I'm using a Panasonic Whisperline fan in a box about half your size and it works great. Then again I'm using CFLs so there's not much heat to deal with.
How would you set up your filter? The problem with mine is that I'm using a carbon tray which sits on top of the fan with the fan unit itself inside the box, which takes up a fair amount of space. I'd link to the setup but it's from another site.
But I suppose there are other ways to set it up. What I like about the Panasonic is that it's so quiet.