Since I have a larger collection, each one is probably handled for about 15 minutes every week. When I had just a few, I'd handle every other day for about 15 minutes.
Each ball has their own personality. Certainly, a little more handling can help to get them to accept handling more, as long as the handling sessions are short and positive experiences.