Do you have a volt meter that you can use to check to see if the output on the thermostat has voltage when cycled on? If not, you can plug in a lamp to see if the light goes on and off when cycled.
Edit... Just looked at the controller on the Ebay link you posted. It is just an simple on/off relay type PID controller. Do you have your heat span set up correctly? Looks like it can be set from 1°-24°. Meaning, if you have the temp span set to 20°, the relay wont engage (energize) unless the temperature of the probe is more than 20° lower than the setpoint and wont disengage (de-energize) until the temp is more than 20° above the setpoint. You need to really make sure you only have your temp differential set to 2-3°.