Find the anthill that is bringing them to the plant and kill, kill, kill, kill! And do it now! Aphids use ants for protection and the ants harvest their excrement for food. They will drain the plant of energy. And the honeydew (the excrement) also attracts other nasty things like powdery mildew and bud rot. Lady bugs or green lacewing larvae help but the ants overwhelm them by bringing even more aphids! Ugh, I really hate those things.
You can smash them rinse them off or fight them with any number of pesticides. It also helps to make sure the only method of getting on to the plant is up the main stalks. Then spread a sticky substance like vaseline around (a few inches in height) the circumference of the stalk. The ants cant crawl through the vaseline so they cant bring in new aphids or take care of the ones already there. Making a thick ring of diatomaceous earth around the base of the plant helps keep them and other nasty little crawlers from getting to the plant to do, that voodoo, that they do sooooo well!
Good luck friend!