When my temps where dipping down into the low 70's (maybe high 60's) I lost some babies from each new litter for a week or so. I was finding them dead without a mark on them so if you aren't finding any evidence there where more then quite possibly not the same issue. I eventually felt confident that it was temp related in my case so I got insulated well enough to stay close to 80 and now I'm back to not finding any dead.
However, my main colony only seems to have about 12 - 15 max and some times a few less. I had another colony that wasn't as docile have 17 first litter but it was that cool spell and only 12 survived the first night. I'm not sure what was up with that colony though as the male looked old when I got him and after neither that mom or the other two looked pregnant in a couple months I just recently feed them off and used the cage to set up a new colony with three of the daughters from that one and only litter with a male from my other colony. They are still a ways off producing but I'm hoping to get the larger litters (they are also a little larger adults) and the nice temperament together eventually.