Unless there's something more than what you've mentioned here, everything is OK.
Step 1 - calm down. Holding for him for a bit didn't hurt him. It's recommended not to hold them until they eat just so they destress. Briefly holding him before putting him in his new home isn't a big deal.
Step 2- walk away from his home. He's probably just curled in a ball to protect himself for now and won't want to move until he's alone and unobserved.
Step 3- Glance back at him in an hour or so. He'll probably have hidden himself.