The fan leaves contain sugars (no, it's not just the sugar leaves) that the plant feeds on to assist flowering. They're for more than just veg.
So yeah, prune selectively. Clear some space for air n light, and damaged leaves, but don't go nuts. Normally you'll be doing most of your snipping...