Yeah, I would let him settle in for awhile (even a few weeks.) I would only really remove him for cage cleaning, brief periods. I know it is tough; I have a female in quarantine now that I'd love to take out and take 1000 pictures of.. but that's not happening!
Oh, welcome to the forum also