I will say I disagree with it being genetics, it most likely has to do with how the breeder fed or raised the babies. Like AK907 had problems, it was probably because the guy had no idea what he was doing, how to present food to picky babies, or had his husbandry allll wrong, causing them to completely stress, not eat, and then never want to eat. It can happen.
Also, for the OP's situation, for both of your snakes - the female was probably not fed often enough, and the male was probably fed something you aren't feeding or maybe he just is a picky snake.
A "enclosure aggressive" ball python most of the time will be due to hunger or you smell like a rat, maybe both
I would also feed in the enclosure, but if he won't eat there, he won't eat there.