3 or 4 times a day? :eek: Wow, I'd be striking at you too if you did that to me!
You've got to take into consideration aclimation time. He's in a strange place with all sorts of strange scents and stimuli and you're yanking him out of his strange new abode every time ya turn around, of course he's going to be grumpy.
I don't know if handling snakes several times a day is your norm but they don't need to be handled more than 3 or 4 times a week, (some would say only once or twice a week) not a day.
Leave him alone for a few days, give him time to relax a bit, then take him out again... to a quiet place...and hold him for about 20 minutes or how ever long it takes him to start calming down and realizing that he's not going to get eaten.

What you can also do, which I had to do with my bull snake, is to take some of the hair out of your hair brush...not a lot, just a few strands, and put that in the snake's container, that way his home smells like him and you.