Yes, it's normal and I think he is scared and trying to flee from you! ... LOL

With time and regular handling, he will become more "tolerant" of your handling sessions. He just has to learn to trust you.

-adam