Gloss will take longer to dry, so, your two day suggestion sounds about right, possibly three. The key will be whether or not you still smell paint when you walk into the room...

I waited a couple days after using a flat latex, which drys a lot quicker.