Do you have more than one adult male per group of females? Usually groups of 1.2 to 1.4 (if the enclosure is big enough) is best. If you leave any extra males that are sexually mature in with them, your numbers will decrease.

Age is another factor. If the breeding females are more than 1 year to 1 1/2 years old, the litters will be small.

Take 1 male and 2-3 females from a weaned group and separate them to replace your older breeding group. Do this with all your breeding groups to keep your colonies young.

Keep am eye on food and water. If they run out for a day, they also may fight or eat the young.

You could also try to bring in new blood. Find someone near you that breeds them. Not the same person you got your original group from. Try to find some colors different from your own to be more sure the blood lines are mixed.

Incorporate a new, just weaned male, from a different source, into one of your own groups of young females.