Leave him alone for a week. Also don't handle him for 36-48 hours after feeding. Other than that, have at it.
I tried the only takin them out when they are cruising around but they only really do that if they are stressed or hungry. I take my girls out once a day usually. Unless you have him out for the whole day, he will have plenty of time to sleep.