is having ________ wrong?
sex, alcohol, tobacco, a tv, a car, an icecream cone,
Ultimately, a Libertarian can not see how it can be wrong to have anything. The government of course will tell you that having certain freedoms (including the freedom to get high) is wrong.
For instance the entire campaign against MJ was a massive collaboration between DOW, Hearst and the Federal Government to make it seem as if users of MJ were some how insane. Never mind that just 40 years earlier the United States was essentially a "do whatever the hell you want as long as it doesn't interfere in the rights of others" type country with minimal government interference (no income tax, no social security, no Gestapo, err FBI)
It wasn't until prohibition (f* crazy Protestant Bitches) that the government some how got the crazy ass idea that it'd be okay (because the people wouldn't object to strenously) to make substances illegal.
Of course, the other reason why the government could not endorse the usage of MJ is because of the impression that it received that drugs some how made people lazy.
If anything can be seen from the last 50 years, its that people are going to be themselves regardless of what they are on. A naturally lazy person is just going to get lazier, while a naturally active person or creative person might end up being more creative or more industrious when using drugs (depending on what they are using.)
Then there's the existence of the Welfare State, as long as people are "endangering" themselves the government would not have support for a welfare state. A Welfare State and Freedom do not go hand in hand. The government must restrict your freedoms to ensure your well-being in a welfare state, where as in a state of Freedom the government allows you to do whatever you want as long as you are not violating another person's rights to do whatever they want.
"Government is essentially the negation of liberty" - Mises
Or perhaps a better way of putting it would be, that I do not expect the government to protect me from myself, or my voluntary decisions. I expect the government to protect me from people that will initiate force against me either to steal my property, murder me or enslave me, all of which are in general equivalent.
To steal my property would be to steal my past
To kill me would be to steal my present
To enslave me would be to steal my future
the end summation of each action is equivalent to death, that is, non-existence.