From a genetic stand point no it does not matter the result will be the same.
From a sex ratio standpoint the only exceptions are Coral Glow and Banana (same thing)
From an investment standpoint yes it does matter, you would not want to waste a 2, 3 or 4 genes female for example with a normal male.