A thermostat and thermometer are two completely different things- a thermostat is actually what you plug your heat source into (not sure if you use a UTH or heat lamp), so that your heat source isn't plugged into the wall. If you get the $30 thermostats, it does automatically what you're currently doing manually: turns the heat source off when it hits the desired temperature, and turns it back on when the temperature gets too low. So it prevents the heat source from becoming extremely hot, which they do very quickly when they aren't plugged into a thermostat. You can set the thermostat to a desired temperature.
All thermostats come with a probe attached, that's their function- you dont need to buy a probe
the probe is placed between the bottom of your tank and the UTH. If you have a heat lamp/above heating, the probe should be hanging in the air under your lamp.
the probe tells the thermostat what the temperature is, which is how it determines when to turn it off or on
Basically the thermostat CONTROLS your heat source's power.