I agree with what the others have said. Leave him in his viv for a week, and make sure he gets a few meals in him before he is handled again
What sort of cage do you have? Perhaps get a lock to stop your roommate from handling him without your permission and when you are not around. It will stop further stress on the animal and you will be able to keep him more secure also. Make sure your roommate knows they are not allowed to handle him when you are not around, especially after he has eaten.
