I've had durian. It smells terrible but tasted like sweet onions to me. Not my favorite, but not terrible either. I'd imagine if you could cook with it and somehow mask the smell then people wouldn't be so against it.
EDIT: is shark fin soup "bad" to eat because they kill the sharks just for the fins?...?