I don't know too much about windex's chemical properties to say whether it's harmful or not. However, according to "Big Fat Greek Wedding" it is a medicine in it's own right. lol.

I used Zoo Med Wipe Out 1 (found at evil petco/petsmart/pet anything...) to clean my cage and everything in it. it's probably more expensive than necessay b/c it's a namebrand pet product, however I have not crossshopped it by volume with other cleaning products. I would just be VERY careful using bleach to make sure that it is well rinsed out and ventilated before re-introducing your animals into their set-up.