That's not the issue, do you have the right to do it in front of/next to someone who believes the opposite of you, that they have the right to sit in their back yard and enjoy themselves without dealing with the smoke that comes onto their property?
I'd really like to see how you actually answer this because this is an example of one of those situations I had in mind when I said what you believe simply can't exist in the real world. There is no such thing as "unlimited freedom" for everyone. Somewhere along the line, your freedom is going to conflict with my freedom, and without anyone/anything there to objectively say who is right, it's essentially you v. me, and throughout human history those situations tend to end favoring the bigger, stronger, more physically intimidating person. I'm sure you can foresee how that usually ends, which is why government exists in the first place.
"If men were angels, no government would be necessary. If angels were to govern men, neither external nor internal controls on government would be necessary. In framing a government which is to be administered by men over men, the great difficulty lies in this: you must first enable the government to control the governed; and in the next place oblige it to control itself. A dependence on the people is, no doubt, the primary control on the government; but experience has taught mankind the necessity of auxiliary precautions." -James Madison, Federalist 51
