I think they do recognize "their people". My son's BRB hates me and will try to bite me every time I take her out but has never once even thought about striking him. She knows! If he takes her out and hands her to me she is OK with that. Silly snake.
His corn snake could care less who picks him up, he's just a dufus.
My IJ's are a bit wary of others and will try to get to me if someone else is holding them, but they don't really care after a bit, it's just for the first minute or so.
I think they know their owners are 'safe' and strangers may not be. Instinct keeps them alive after all.