Exactly what she said above. It would cause more harm to move them out than it would be to just keep them in for 3-4 litters and then give them a rest for 2-3 weeks. Then, back in with a male for another go around.

If you kept moving them in and out, back and forth, they are going to get stressed, possibly eat their babies/each other, not produce well(intro time plus time it takes for her to even go into heat, then to even get pregnant might be months), plus other things. Harem breeding is where they are left together for an amount of time, then removed to rest and switched out with others.

If you keep removing them and putting them back with another male after their litters are weaned, they are going to have to get reintroduced, which could take months(and has) and then you've just wasted that time that those mice could have been producing for you.

With rats and the harem breeding(at least in my and others' experience), they stress out, steal each others babies(babies get killed or hurt) and hoard them, become used up too quickly. It can cause a drastic decrease in their lifespan. Something I haven't noticed with mice.