Don't use a sponge! They harbor bacteria like nothing else, and it would just add to your chores to disinfect it constantly.

Why don't you have a bucket or 'dumping container' of some sort and a jug of fresh water with you when you change water? That way you can do it all right there and just dump out the bucket when you're done.