It certainly won't hurt to leave him alone. If it were me, I'd give it a week or so and see if he gets around to rubbing it off on his own. If not...and if it bugged or concerned me in any way, then I'd try the wet washcloth in a pillowcase trick. Works like a charm.