You can feed him in his cage, he's not going to mistake your hand for food or whatever line they told you, as long as your hands don't smell like mice at all times.

If you got him 3 days ago, you probably wanna leave him alone for a week or two without handling unless cleaning the cage. It's cool that he's eating so readily, but he needs time to get adjusted to his new environment.

Don't handle him after feeding, it could possibly cause regurgitation.

Edit: Beaten to it,