It just means your soil was colonized by fungus. It doesn't necessarily mean you're overwatering, though that's possible.
Pull it out if you like, but even if you do that it's just the fruiting body of the fungus. Most of it will still be living in the soil, and with your plant in there, its too late to try and sterilize the soil.
I doubt it will hurt anything anyway.