Are you sure you're getting the prey hot enough? It should be roughly 100° when offered. And keep in mind, it cools fairly quickly.
As for the substrate, don't sweat it. They have been swallowing dirt for thousand of years and are just fine.
And are you sure there was substrate in the mouth and not just realigning the jaw?
Hold off on offering for at least five days. Offering too often can stress the snake and lead to more refusals.
Oh, and next time leave the rocks, hides, etc... in the enclosure when you feed. You want it to seem as "normal" as possible. Even something as little as moving that stuff may be enough to stress the snake enough that he refuses.
Think of it this way, they are ambush predators who hide and wait for prey to be within reach. They're usually in tight quarters when they strike in nature. So as keepers we try to mimic nature the best we can.