Agreed completely in this situation. Brother, Friend, Family member.
I would even go as far as to do this in a very public place, or instead of bringing one person, bring a few.
If anything, the fact that you have people backing you up with scare him, and his tail will be tucked. The fact that you have people backing you gives you the upper hand, and you establish dominance.