I live in a small suburb of Milwaukee (Bayside) that is incorporated as a Village. The Village only has 600 households and most construction was done in the '50s. A large percentage of the houses are still occupied by the original owners, and I bought from the estate of the original owner as well. The Village seems to run like a '50s version of a HOA. They have managed to enforce and keep covenants active for 50 years now.
Not sure if it is legal, but I guess I don't see the point of being stubborn either. I am going to call and ask why it is being re-assessed 1 year after it sold for 20% less than the previous assesment. Perhaps they will back off, but if not, I'll let them through.
Main reason I posted this was because I have never experienced this sort of city inspector doing a physical walkthrough before and wondered if a non-permitted improvement would cause major hassles. Like Big said though, they probably just want the money.