Only gonna be an issue if you loose power and aren't around, then you might get some stinkage out of the intake port(s). Otherwise the negative pressure created by the exhaust will keep all of the smelly stuff directed through your exhaust filter.
Also, while the HEPA style filters are very good at limiting junk getting to your grow, they do need to be watched cuz' the finer the filter the easier and quicker they can fill with dirt and loose capacity. Best to change them out when they still appear clean since they are trapping a lot of stuff that's smaller than you can spy.
A cheap alternate is a block of aquarium filter foam. Called reticulated foam, it is referred to by its PPI, which stands for pores per inch. Generally 30-40 PPI makes a very efficient air filter (they also use the stuff for a lot of small engine air filters) that you can take out and wash to re-use over and over. Nowhere near as fine as the HEPA approach, but an inch or so should keep the bugs and mites out.
If you want overkill, try a shielded U.V. bulb on the HEPA intake filter. It'll kill the bacteria and viruses that get tagged by the filter. It's what they use in burn rooms. Whatever you use just be mindful that the more shit that you make the air travel through the harder your fan's gonna have to work and the lower the ultimate CFM you're gonna realize from your fan. This is especially true with computer fans since they are designed to be run in very low restriction environments. You can knock the low pressure fans down to about useless with a compacted carbon bed and a restrictive intake. Also the quieter fans are also usually the lowest pressure/slower speed ones.