The royal familiesGranted they did that generation after generation, but from what I've seen on the history/discovery channel it wasn't really physical deformations that were the problem, it was the genetic health disorders that started popping up.
To the OP, generally one time of sib to sib is fine, and mom to son/dad to daughter is fine, past that people start to get their panties in a bunch.