IP Flags removed by Annapolis Harbormaster!

Well, so much for freely flying the beautiful Island Packet GOLD flags as a PRIDE of OWNERSHIP from our boats while in the mooring field! The Annapolis Harbormaster boarder our IPs today AND LOWERED AND REMOVED OR IP FLAGS FROM OUR BOATS! I about flipped out! The harbormaster actually told me that if I hoist my OWN IP flag, one that I paid $150.00 to have made by the Sail Bag Lady 3 years ago, that I woul be escorted OUT of the Annapolis Harbor! We are NOT allowed to fly an Island Packet Flag from the mooring field. Here are some photos of the harbormaster boarding my neighbor, Voyager C, an IP35 from Nova Scotia! The question is: Is it legal for the harbormaster to board boats and take down flags that are flying from our boats?