I would wait to handle for enjoyment until he's eating three times for you: I like to get my snakes established before messing with them.
He could have been surprised when you reached in to get him: I would feed inside the enclosure as it's a lot less stressful on the snake. He won't get "cage aggressive"