A good couple years ago, my mother was getting dirty looks, verbul abuse and threats from a male neighbour, whos in his 30s, a couple houses down from me (the fencing out front isn't very high so you can see start over here). The first couple of times, she told me don't do anything you'll make it worse. So I agreed, didn't do anything.
One day, as my mother and I was getting in the car going to the pet store to buy some dog food and fish flakes. The man was talking out the trash, he saw us, started to verbally shout abuse at us. That moment, I flipped, completly lost it. I got out of the car, started to walk down the street the man saw went inside his house, I went to his front door banging as hard as I could for about 5mins when all he did was pop his head out the front window and told me to "innapropriate language removed by staff".
I went out to his truck and picked up a metal poll, this guy was some sort of builder, I swung the poll smashing his couple year old Land Rover. He instantly came running out of his house. I throw the poll away, pinned him to the wall and told him to stop with his abuse, all he did was spit in my face. So I put him in hospital with a broken nose, couple bruised ribs and a broken right arm. He hasn't even dared to look down towards my house now.
But this got me in alot of trouble. Keep doing what your doing, get a lawyer. Don't retaliate.