Your neighbor sounds like a cowboy. Maybe I am just too security oriented but I would NOT have unpacked all that stuff during a time when people could see it. People aren't stupid...if they see you bringing a shitload of soil or gardening equipment and you dont have a visible garden, it raises eyebrows...same with alot of anything, especially in this economy.
I would try to insulate my self from his business, because if he is taking this kind of risk already, who knows what kind of danger he will bring to your op.
Personally, even though I am legal, security is top concern. I take care to not stand out to my neighbors, I have taken equipment in @ 5 am, just to keep my opsec.
My .02 is to find a new spot...Sounds to me like your neighbor has cowboyitis...not good.
CK
Edit: They DID get caught!!! See everyone?? Being a cowboy gets you rounded up!
CK