Not sure about the legalities, but FAA regs clearly state they are in charge 'from the ground up'.
And this is a classic case of drone hater vs drone enthusiast. My question is, if the drone operator was standing on his rooftop holding a video camera, would the neighbor have shot him in the name of privacy?
If your outside, there is NO reasonable expectation of privacy. If you do expect it, you're a fool.