I would believe you need to do it right now, that size plant still has a small root system for that size pot. You can take them out wash it out break away most of the soil u can till you get something like a soil/roots ball. Then you can replant them into the soil with perlite. This may shock...