I don't pull or cut the fan leaves until they are completely yellow and hanging limp, the other exception would be diseased or mold I guess. The plants use the energy stored in the leaves to make the flowers, so cutting them off decreases the available energy to the other parts of the plant mainly the buds. If you have an abundance of fan leaves blocking light to buds try moving them around to allow the light in but I would strongly suggest leaving them attached to the plant, nature put them there for a reason.
Happy Growing