I wouldn't try so much to pull it off, more to try and lightly brush it off, make sure you keep the stem steady with your other hand too, i've pulled one out once trying to get the shell off, and yes you must pull it off sometimes they just don't break free of the shell.