We never had that problem when we bred mice. We did use fancy colored mice though, not sure if that makes a difference or not. Personally, we found all the albino mice to be kinda tweaked and they were the ones we had problems with when it came to getting along with their cage mates. Just my 2 cents ...
Robin