I'm going to second (third? seventh? SIGN OF THE DEVILTH?) the supervisor comments. I think if this continues you really will have to at least mention it to him. He'll laugh or roll his eyes, but at least he'll remember if it comes up. Then if it keeps happening and he keeps laughing and brushing it off, you may have to go one over him.

And like my friend says: always start with and try to stick to the "I feel..." or "it makes me feel..." statements. Those really do work.