If you are using a dimmer instead of a true thermostat, then that plus the room temp change is absolutely the culprit. A dimmer won't adjust to external changes, it just allows the power through that you've adjusted it to, which is why dimmers should only be a very short-term solution to managing your heat sources.