A lot have to do with the way they are house, security is really a must with mice.

Dark tubs are better than glass tank or clear tubs it allows them to have more privacy reducing the stress level.

Good diet is also very important.

Often it also gets better with second generation has they are less skittish than the one from the pet store.

When mine give birth to 17 babies 3 weeks later I still have 17 babies, I rarely lose any and when I do they don't even eat them they bury them just like my rat do (go figure I must have weird mice)