By now he's probably afraid to peek out...any minute the "roof may lift off again"!?But BPs are ambush predators that prefer night time, & maybe you just aren't up
late enough? or he ducks back in when you come in the room...he sounds fearful enough to be doing that. Snakes can feel our footsteps, especially if you don't have a
concrete floor. You probably turn a light on too...
You know, we hear that a lot..."tamed down before he gets big..." but snakes just don't grow that fast. Many wish they would, but they just don't. You have to do things
at a pace your snake can handle...he is still a wild, instinctive animal even if captive-bred, & he has no concept of time. It just works better if you don't push them...you're being the equivalent of a "stage mom or dad".Put him first, try to read his moods & needs (this gets easier) & with any luck, you'll be in sync.
When a snake refuses food, he may also be going into a shed. You might wait at least a week or two (to see) before offering food again. Then (if he's not in "blue") try to
entice him with f/t prey wiggled very slightly just outside his doorway* (of the hide he's in). But whether or not he's going into shed now, I can tell you from all that you've described that you're over-stressing your snake, & accomplishing the exact opposite of what you want: a confident snake that doesn't feel threatened by you & is fine to hold.
*And for best results, make sure the prey is warmed to seem lifelike, right before offering. BPs rely on heat-sensing pits the most.