When left alone long enough, the air temps in the box will maintain the same temp as the air outside unless you're using a heavily insulated eggbox. Though iirc the eggs will start to warm up more before they hatch.
Some breeders will use a dummy box, a box where the probe can sit and you get the general temps for the incubator. This really only works for incubators where the air is circulated because the temps will vary from top to bottom
Because I was only working on a single clutch, I was able to put my t-stat probe directly into my eggbox and cover it back up.
I'm looking at building a better incubator for next year to when I'll be hatching up to 6 clutches. One with a circulation fan and at least 3 individual digital thermometers to measure temps on different levels.