That is one of the reasons most people stop using driftwood in BP tanks at all. If the mold doesn't come off, you have to replace the wood.

What substrate do you use? It shouldn't be so moist as to cause mold to develop... the key is to get the air humidity up to 50%...not the ground. Substrate should never have noticeable moisture in it.