Leave the probe where it is. You need it there to monitor the temps of the heat pad. The reptitemp doesn't display temperatures so you need the accurite there so you know you have dialed it to the right temperature.
You do NOT need to remove the UTH to use a thermostat. many people recommend putting the thermostat probe in between the UTH and the glass, but it isn't necessary.
I usually just tie up the extra cord and put it behind the accurite unit. You could run the extra wire outside the tank though, so there isn't a bunch of wire all over the tank. That would not be fun to clean if it got crapped on.