I'm an organic grower. I make my own Super Soil. My base soil is Roots Organic, which sometimes has a lot of fungus gnats in it. I'm assuming that your flies are fungus gnats.
These gnats are harmless as adults, but their larvae in the soil will eat plant roots until they emerge from the soil as adult gnats. So you definitely want to get rid of them ASAP.
There are two ways to safely get rid of them. I've used both, and both work.
1. Diametrius Earth (powder)
This is a fine powder. It is NOT a poison. It's a perfectly natural product that's safe for you, pets, plants, etc. It kills fungus gnat larvae in the soil by cutting them up as they wiggle through the soil. Although the stuff seems like a fine powder to us, it's actually like a bunch of jagged razor blades to the small gnat larvae. They get their asses sliced up as they try to emerge from the soil, before they can lay eggs. The method of use is to simply mix some of the powder into the top 2 inches of soil each time you transplant.
2. Tanlin Concentrate (liquid)
This is a sort of liquid form of Diametrius Earth. You simply add a few drops to your feed water the first 2 waterings after a transplant, or whenever you see gnats.
Again, I've used both methods and both work like a charm.
Both products can be found on Amazon.com.