I have thought about this many times too. I think the main thing here is not numbers like mentioned before. It how well the animals are cared for, like being fed/water, vet care, socialization (when that applies), adequate shelter, clean living space. With cats and dogs, are they spayed/neutered to prevent over population?
I see nothing wrong with people owning as many animals as they can care for properly.

However, with hoarders there is another problem. They don't have the ability (or mental capacity) to seek help when needed, they feel like they can't find homes for their animals. Some even keep carcasses of passed animals in freezers, or other storage places because they have the inability to part with their animals that they think they love so much. Now these are extreme cases that I have seen, and its not pretty.

Where say most of us, if we come down on hard times, have more than we can care for. We are able to pass a long our animals to others in the form of a sale, or even giving away animals because we are more concerned for the animals health and going to a good home.