just leave them be. i dont take the male out, but some do.
you dont have to take them out till its time to feed them off or their weaned and there is a new litter of pinks(sometimes the weanlings will eat the pinks)
males will get bigger faster once weaned so if you need adult mice you might want to think about that when picking out feeders.
mine all go into a single tub when they are weaned and i feed from there.
right now i have 40 females across 5 tubs. im still playing with my numbers but i have one tub with 1.11, one with 1.9, and the other 3 have 1.7 groups.
im using the ars 2020 mouse grow out levels.