I grew up in Michigan, and don't find 'ma'am' to be in any way an offensive thing to say.
I have heard some ex Army guys saying "Don't call me Sir; I work for a living", though, lol.

I always hold the door for people behind me, regardless of who they are--the older guys appear utterly bemused by this, so it's fun.