Yeah, but just letting it sit out evaporates chlorine and thats about it. You can still have water with a ph of 8.0 or 5.0 for example after it has been sitting out. You want to find a medium between your soil and water. If you soil is 5.5, you want to get it around 6.5 -7.0, so you need to water it with water that has a ph of 6.5 or 7.0. But lets say your water has a ph of 8.0. You would need to lower it to 7.0 before your soil is balanced, otherwise you might be increasing your soil's ph to 8.0 which is too alkaline. pH is very important, man. It's easy to fix too if you have a ph control kit and a soil ph meter.