Balls stress easy. Moving them can pretty much make them not hungry anymore. Or they want to explore and find a place to hide.
The point is you want to stress them as little as possible.