You're thinking Thermometer here, which you would set up just like you describe.
A thermostat would have the probe attached to the UTH, usually on the outside of the tank and taped directly to the heater. You would then set the temp on the thermostat to keep it from getting over 93-94 degrees. The hot side temp would need to be measured on the glass, inside the tank.
On the tanks I have both an UTH and a red light on, I only plug the UTH into the thermostat and leave the light on all the time. The light, if isn't over 50W or so, won't get hot enough to cause a problem and the thermostat can them regulate the UTH temp.