I grow in ground and don't have any issues with ants. i'm in the pacific northwest and we have big black carpenter ants and small fire ants which ive only seen the latter effect my plants by farming aphids. If I do find any ants farming aphids literally one spray with neem oil they are history.
just watch your plants shouldn't be to big of a deal. This is exactly why I plant in-ground or have my pots raised up to where no bugs can bug them. I'd suggest getting something to raise them off the ground a bit. Pallets work wonders
I mix in Gaia 4-4-4 in all my soil outdoors when I plant, I also top feed every month or so with it. Although I do water with other nutrients as well such as Alaska fish fertilizer. Plants seem to love it. had many 10+fters last season