In my experience, when you handle them and they move around alot, it means they're trying to get away. If you give him a chance, he'll find a nice little spot to curl up in and never come back out, in this case it was your shirt, and he probably chose that because it was close by and warm.

Things that i've noticed mine do when the're stressed are:

Excessive, quick movements, in different directions I.e, when he's in your hand, and is elevated, he might try to dart in a direction, realize he can't, try to dart in another direction, realize he can't, etc. until he finds a place he can go. If he can't go anywhere, he may ball up right there.

Look at his respiration rate, if you notice quick, heavy, spaced breaths, he's most likely scared. And during this time he may make some popping noises.

When they're startled, they may also freeze in place, hold their breath, and stop flicking their tongue. My interpretation of this is that you have triggered the brain to go into the "fight or flight" mode, and after a few seconds they will decide to either ball up, run, or strike (which is VERY rare).

If he's just kinda wandering around, flicking his tongue alot, and isn't jerking to go in different directions, then he probably isn't in any actual stress. But keep in mind that these are snakes, and they don't "like" interactions with humans, they simply tolerate it, and I think it would be safe to say that he would much rather be crammed in a small, warm, dark hole where he knows nothing can get him.