UPDATE : After using neem oil for a week, it haven't been much of a success. I brought the leaves to my local gardening supplies and they looked at it under the microscope. It was red mites (Eutetranychus africanus) and got some foilar spray to get rid of it.