There you go. You're doing it right.
This is a visitation issue, not a custody issue. It's much, much harder to lose visitation than it is to lose custody. At least in AL, you have to get the courts involved BEFORE you can keep a kid from the visiting parent, even if it's due to safety reasons. If she's not involving at least DHR, she can't keep him from you. The burden of proof lies on the parent claiming unsafe conditions.