I run a 150w HPS in a small tent (2x3x5), and it does get hot as hell. Like xtsho said, a sealed hood with an inline fan pulling air directly off the fixture makes all the difference in the world.
As far as using CFL, you CAN use them, and grow a plant, but you wont get nearly the results you'd see other people getting. LEDs would probably be best for you if a small HPS is too hot, but you may still want to invest in an inline fan to vent heat out of your grow space (plus they're useful, paired with a carbon filter to remove unwanted smells leaking out of your grow area)