I generally don't handle for extended periods. Just for wellness checks, or for VERY short periods.
But it I have a fairly large collection of animals. Quite simply, my time goes to the animals that need or enjoy being handled, with those that only tolerate it being much less.