If your growing outside at this time of the year I presume your either in Australia or NZ, and while your plant is fine now and the bottom leaves as stated above always end up gone I would not be overly concerned with that, although a lot of people here are probably from the USA or Canada and don't know the level of pests we have is incredibly harsh on plants. You can go from having a 3-4 foot plant looking great to a pest ridden half dead monster the next day.
You want to get preventative in veg period prior to this occurring, the way to do this is as follows:
DONT spray your plants as stated with bug or pest sprays unless you already have an infestation, they are a last resort. If you do need to spray do it with an organically friendly pyrithium spray that is much less harsh than other brands.
Put Pet and children approved SLUG and SNAIL pellets sprinkled all around your plants, if wild animals eat them it will not affect or kill them (you don't want this), but it will prevent any crawlies including catepillars from coming near your plants. Put more out once a month as they will tend to absorb into the ground with rain etc and become less effective.
Use neem oil on your plants in a weak foliar spray as a preventative or more aggressive if you notice pests becoming an issue, if you don't know what that is google it and its application, you should be using this all through veg and into flower for at least a few weeks.
Put fly papers around the area of your plants, 2 to 3 is fine at a time and they are not overly expensive, this will prevent most of the flying bugs.
If you use these methods you will probably notice that you will have very few issues with any types of bugs your entire grow.
If you have animal problems like deer foxes etc that is another issue (your plants aren't eaten so Id presume you don't in your area).