I'd wait til they were weaned if you are uncertain, but why do you keep the male in a separate cage? We keep our males in with mothers and babies. They help care for the babies as well and take a lot of strain off the mothers. We've never had a problem with male rats killing offspring like you would have with other rodents like mice, hamsters, gerbils, etc.