Honestly, I don't think the exact mechanism of inheritance has been determined. Basically there's something a bit odd going on with the inheritance which doesn't make it act like a typical sex linked gene but it is somehow related to sex. Generally males who had a male banana parent will only have male babies come out as banana. A male with a female banana parent will generally have only female babies come out as banana. A female banana will have both male and female bananas.