Ah, I have nine wee babies now, barely over a month old. They're all pretty defensive though I can usually pick one up without getting nipped, usually by using one hand just out of range in front of them while the other scoops them up from behind. Once in-hand I try to make a "hide" out of my hands for them, they seem to relax and settle faster. After they relax I put them back.

The suggestion to cover the enclosure so he can't see you is also a good one.