In my opinion I wouldn't do anything brash right away based on their first litters. I recommend reviewing all your husbandry and then giving them another shot.
If they fail to produce a second time then I'd say feed them off and try some new stock.
Also make sure to hold back some females from the healthy litter.