It should be okay to hold him until his eyes cloud over. After that, they get a little skittish and nervous.