If there's only one port, and you connect a fan to this port to pull hot air out of the cab, the air has to come in from somewhere. With no glass in place, the hot air will be pulled in from your grow space, through the fixture, and up and out through the ducting and exhaust fan. If you have glass in place, where will the air be pulled in from? If the glass is sealed tightly in the fixture, and you have no other openings, you'd just be creating a vacuum in your light fixture.
Most air-cooled hoods have an input and an output port (usually on the sides). The glass seals the fixture, so air is pulled in through the intake port and is exhausted (along with the heat generated by the lamp) out through the other port. There has to be some kind of opening for air to be pulled in, if you're going to exhaust warm air from your lamp. With no glass in place, you have an opening.
Maybe your fixture has some vents on the side? That might help a little, but still would probly restrict air flow.