Well, I found out what happened.
Public Works for the town removed it. They claimed they sent me a letter saying I had 10 days to move it or it would be removed by them. Evidently they thought it would be in the way when they come to install the new curb on the street.
I explained I never received a letter but that didn't matter to them. They claim to have sent it and that gives them justification to come and remove it since I didn't within the time frame they supposedly gave me.
When I told them that the crew that removed it trampled all of the flowers, they basically told me too bad. The crew was instructed to remove the stone and they cannot be worried about flowers when the homeowner ignores the letter that was sent. I NEVER got a letter. Very, very unhelpful and unapologetic people work for the town. At least now I know what happened but it still is a pretty sad that my tax money paid for these buffoons to come out and trample my yard and take stuff and I have no recourse.