To keep fruit fly numbers down:

Leave the dish in overnight only.

I have had this problem as well, and in fact recently got rid of my dubia colony because their food and/or poop attracted insects like crazy.