If you are using vermiculite instead of perlite remember to keep it damp.
It is almost impossible to get vermiculite in North America without asbestos in it. If you keep it damp and do not have dust clouds when dispensing or disposing of it though it shouldn't be a huge problem.