Man if it never occured to you that you were "sad" in the first place and took your mate to say so then perhaps he's the one who's sad and your just doing what you enjoy? If it's causing him no harm why should it bother him, and if your enjoying yourself it shouldn't bother you. Keep toking!