if you see little gnats flying around too, it could be fungus gnats. The adults don't harm plants, but they lay their eggs in soil and as the larvae feed they can harm the roots of your plants- I'm guessing that they are consuming beneficial fungi as well.
I had that problem and treated...