Keep something that conducts heat well in the empty space to hold it in. Most people use water bottles. If the power goes out, it takes significantly longer for the water to lose the heat than the air around it.