nah I've had a grow in the past where my a plant of mine ended up with root rot which is a systemic infection that caused buds to start turning brown from the bottom of the plant and worked its way up, luckily by that time it was about time to havest, but what I ended up doing what making it into butter after dried as long as u let it boil for about 30min, I even throw a shot of 100 proof vodka in my butter recipe, anyways after that then strain the plant material out of the liquid and u shouldn't have to worry, any mold spores or w/e should be killed by the boiling water.
This is a pic of one of the harvested buds one side was green the other brown as u can see, I cooked and ate it after and lived Lolz.