The clicking is the thermostat turning on/off. The alarm is a beeping noise.
From what you observed with the lights it sounds like the bottom thermostat is set to proportional and the top is set to cycle on-off. Try setting the top thermostat to proportional and make sure the variance is +/- 1*F.