odd white mold spots 1 to 2 inches below soil line. help!

I pulled a male up yesterday and noticed these small white mold spots just below the soil line on the stalk. The plant was vigorous and healthy, and the mold spots wiped away cleanly and didn't appear to have any affect on the skin of the stalk where the mold spots were. The roots appear to be unaffected as well. I've been growing outdoor at this spot for 13 years and have never encountered this. I scraped away an inch or 2 of soil around the stalk of some other healthy plants, and some have this mold and some don't. Yet there is no noticeable difference between the plants with or without the mold. They are all in the ground with very similar soil compositions. Could it be powdery mildew? A few plants in my garden have very minimal pm spots- almost unnoticeable- but I've been treating it with a weak solution of neem oil. Could it be a harmless fungus? I'm at a loss. Any body know what this fungus is and how to treat it? Do I have to worry about it even? I'm scared it could destroy my whole garden I've minimized watering in the mean time. Help please!!!
Thank you for your time and consideration!