If you can barely open the door have you considered changing your exhaust and intake to make it a negative pressure instead of a positive? (seems like your intake is larger, when the exhaust is larger it allows an neg. pressure) Obviously this is not always possible, but what if you just put your intake fan on your exhaust and your exhaust fan on your intake? Does that make sense? You could also (in theory) entirely reverse the flow in the room, intake threw your current exhaust and exhaust threw your current intake? Assuming that it is an inline fan i would agree with mounting it on bungees for the noise, or if possible kick it entirely out of the room. Using insulated ducting helps to isolate noise also.
I just reread the title, do you have an inswing (into) or outswing door on your room? I assumed inswing, if it is inswing you have positive pressure not negative, outswing then negative pressure is the issue and most of my post is stupid, hadn't smoked yet and now i have, thats funny, im smarter high.