Personally because you have no idea of their age, I would wait until the girls are just about to pop, then pull the male and feed him off. Then keep the best looking babies for your 1st real breeding group.

When first starting a colony out, I believe it is best to start with a young group so you know what you are working with.

The breeder had to know what he was doing. By what he told you it sounds like he knows the male probably isn't the father of the pregnant moms... and that could end in disaster.