99% of the time it's nothing to do with the males, it's the females not wanting to breed.

IS he like this with all your girls? What's his age and weight? Have you double/triple checked his sex by probing him yourself?