There is no need to worry unless the snake is losing a significant amount of weight, or there are signs of illness. Sometimes it works to offer smaller/different prey, but sometimes that just stresses them out even more.
I would say at this point since you've tried so many different prey, go back to what he usually ate, and try once a week with that, with no handling in between. You can also try putting the snake is a smaller enclosure (such a s a tub) to increase security. Best of luck. You may want to read this thread as well
http://www.ball-pythons.net/forums/s...ad.php?t=64619