are you guys sure thats the right location for them? u sure its not a female plant that got fertilized and a seed forming and not the male 'balls' or whatever?
i ask cuz i dont know that much about sexing, i usually wait til im 110% sure its a male before i remove them, and still haven't gotten any hermies.
--
if it is the male genitalia or whatever, then it becomes almost perfectly round, then opens up, releasing the pollen. i kept a male around to make seeds last time.
(my memory isnt so good, go figure, but that's what i recall)