So, if a culture features throwing virgins into a volcano to make the crops grow, and a piece of paper like the Constitution allows slavery, everything should remain as it is? It then becomes okay to force people to do things even if those people are leaving you alone?
The constitution has no more authority than a coloring book (if people claimed a coloring book was an "official document") , in that neither have been agreed to by the people that either document would claim to have authority over. Have you read Lysander Spooners essay regarding the constitution?