You could be handleing him a little too much. 2-3 times a week is enough to keep him tame but will not stress him out. You definately don't need to hold him for 36 hours after eating or during his shed cycle. Give him a little break from holging him and see what happens.