I only give my males 2 females per season. This keeps them going strong all season and then some.

Also feeding is essential too. If a male is young and stops eating during breeding, I stop his breeding till he eats again.