Your Animal's Rights vs My Animal's Rights

AKA My current challenges walking my dogs...

I have been trying to take Eva and Ares for walks in the mornings.

1. Because it is good for them  
2. Because it is good for me  
3. Because it makes them calmer and less destructive 

But I've hit a bit of a glitch along the way. A few blocks away is a Rottweiler (before anyone jumps I love Rottweilers and as a Pit Bull owner I'm defensive of other dogs who get a bad rap I'm just including it for size reference.) who lives outside all the time, all day every day. I actually feel bad for him because it gets hot and cold in Virginia.

They have a waist high wrought iron fence around their yard with a very poor gate at the back for the drive way. If he wanted to he could jump the fence or he could easily break through the gate. He hasn't so far. He isn't fixed so he is territorial. Every time we walk the dogs past him he charges the fence and puts his paws up on the top, etc. So I decided to avoid the situation...or so I thought. I decided instead of going straight I'd turn down the street and walk down around the block to avoid his yard. Well walking down the block across the street from his yard he still charged the fence and tried to get to Eva. He went to the back gate and nosed at it but that was it. But I'm very worried about him getting out and I think because he is so territorial it would get ugly fast and I wouldn't be able to do much about it. So I've decided just to completely skip his block all together when I walk to avoid the situation.

I worry about him because I think he is neglected which is another issue for his energy. And as I said he is territorial because he spends all day in that yard. Other neighbors have the same issue with their dogs. Fortunately there hasn't been a serious issue.