There's nothing wrong with handling during the day. That's when I do it, especially with my rainbow boa. She's a sweet girl and normally very calm but when I try and handle her at night she gets a little 'hyper' and likes to try and fling herself out of my hands. She's pretty quick, too! lol.

Maybe try and get into the habit of warning your snake that you're about to handle them? I always give my bp's a stroke on the back before I pick them up. If you're uncomfortable doing that with your hand, maybe get a snake hook and try that way