This is a facile and narrowly viewed hypothetical argument. First off, the labor is all over the market in which land values increased. Secondly, it gives absolutely no indication as to how the value increased, if something was built on it or if obstacles were cleared. Thirdly, someone has to do the work of selling it, buying it, maintaining it and monitoring market value. Last but certainly not least, no new wealth was actually created. The value which was derived is artificial, and based upon perceived scarcity brought about by changes in supply and demand. In essence, the land is the same. Artificial value is not wealth and can vanish as a bubble pops.
This is exactly like your Wall Street example which failed. Someone bought something and sold it for more. The increased demand which led to the increased value is still either based on labor or is an artificial bubble.