Comes down to this. Each animal is an individual. Things bother some and not others. Just like all other living things. You will know when something bothers a ball. Heavy breathing is a good indicator. A sudden change in behavior is another. I have four children. The snakes don't mind them. There are two of my snakes that are off limits because of attitude problems. I did not raise them, but they are coming around. Most problems with balls have to do with husbandry issues or infections from outside sources. They are actually very easy animals to keep. Proper handling is not going to stress an animal that is not already having issues.
Also, when looking for advice, try to determine how much experience the one giving the advice has. There are those who like to just repeat things they have heard and may only keep one or two animals. What they say may be true for their animals but it may not be true across a larger population. There are also experienced people that may disagree every now and then, but that is because there is more than one way to skin a cat.
This is what makes this board great. Things can be discussed. I personally have learned a lot here.