Are they the dial type? If so those are known to be inaccurate and have a very short lifespan plus if you are using a UTH they can't tell you how hot the substrate is (UTH's do not effect air temperature) these cost $12 at Walmart and work great (measures both hot and cool side temperature as well as humidity)
Turning off the heat lamp will help raise your humidity BUT you can only do that if the air temperatures stay above 75 degrees. Otherwise it will need to stay on 24/7.