I find I must politely disagree

You should never keep only one goat. You'll find that goats kept alone will be destructive, bored, and intent on getting out of their pen. Goats are extremely social creatures, and must be kept in pairs at the least.

It's awesome your goat stays around, but it doesn't always happen so well, and there are dangers to a loose goat. For example, I once had a goat I found on the road, apparently lost. I didn't bother to put him in the fenced goat yard, I just let him run on our large property, as at the time, we didn't have any other goats (my parents kept a small herd of meat goats for many years). Sure, for a while he stuck around my horse, but eventually he became so lonely he left. He made his way all the way down the road, living with one neighbor after another until he found a neighbor with some other goats, where he stayed.

Sure, goats will do everything in their power to get out of their fence. If they can. The point is to insure they cannot. Which can take a bit of time and effort, but can be done! My goats got out a couple of times, pinpointing weaknesses in their enclosure, and after those were rectified, they no longer get out. Loose goats are in terrible danger from loose dogs, cars, and other people.

Anyways, your goat is absolutely darling! I love that coloring. There is no doubt he will get out of the fencing I see there, you're quite right they don't respect fencing like a horse will! It's great he sticks around even when he gets out. If you'd like to discourage him butting you, you can try a squirt bottle, I'm sure you know how much they hate to get wet. As for aggression in goats, this is less about the sex of the goat and more about how they were raised. I've had all three sexes (bucks, does, and wethers) and have experienced the extremes in all cases. Bucks however, DO have a smell and some very unpleasent habits. A doe and a wether are always a good bet, or two does, or two wethers even, if you never want to have any kids.