If you are certain you don't have worms, then you are doing the right thing. Keep plucking the effected buds and let her finish. Be careful not to handle other plants/buds after handling the mold as the spores could spread.
Its nothing to panic about, those in cooler climates deal with it regularly as the morning dew is heavy with the huge temp changes this time of year.
If your plant isn't already spread out well, try tying branches down and giving the plant a more open structure to allow more light and breeze access to the nugs.
Good luck! Keep a good eye on things, don't over react, and enjoy an excellent harvest!