I've noticed this before, it's once you have them in breeding groups is when they get all nasty.

I keep all my non-breeders together based on sex. All males together and all females together. Once you mix them up and put able to breed males and females to together is when it hit's the fan.