That could be it. I'd give him another week or more to settle back in. Keep handling to a minimum again, and then after that week or so, feed him like you always do. He'll most likely take it just fine.