Mushroom spores can't colonize a living human body, although that would be mighty creepy if they could... but you breathe in thousands of mold spores every day, and those don't do anything.
I've heard that you can smoke fly agaric (Amanita muscaria) and that smoking it removes some of the side effects. Anyone know if this is true?