sounds like it could be larvae, the flying gnats are annoying but they wont hurt your plant, there larvae is what you need to get rid of
Figure 1. Fungus Gnat Larvae
Life History
Fungus gnats are generally found associated with highly organic soils or potting media. The adult does not damage plants and normally moves around on the surface of plants and growing medium. They are poor fliers and generally considered a nuisance. Eggs are laid by the female in the soil. After 4 - 6 days, the eggs hatch and small larvae or maggots feed on small feeder roots and root hairs. They may also tunnel into the base of cuttings and feed on roots and foliage close to the soil. A flimsy cocoon is formed on or in soil where pupation occurs. After 5 - 6 days, the adults emerge completing the life cycle. Under greenhouse conditions there may be many generations per year.
Damage
Damage is caused by larvae feeding on the root hairs and fine feeder roots of bedding plants. Other damage may include tunneling into the base of cuttings. Plants attacked by fungus gnat larvae will lack vigor and will have yellow leaves. Roots will have small brown scars on the surface. Both adults and larvae have been implicated in vectoring the spores of soil pathogens such as Botrytis, Fusarium, Verticillium, Pythium, and Phoma.