Yes. it is possible to stress out a BP due to too much handling. I would keep the sessions short (10-15 minutes). Keep in mind that they also need to stay within their comfortable temperature ranges.
This is not to say you should not handle him. By all means, enjoy the little guy. Just don't stress him out.