I've never had more than a couple of amber trichs until after they swell. At least on the calyxes. If you're checking the trichs on the leaves you'll definitely cut early.
As to clear trichs and potency, I read about a researcher who found that there were actually higher levels of THC in clear trichs than in cloudy, but I can't remember the name. I was looking because I had a light fall on one of my plants at the 5 to 6 week mark, it snapped the stem clean off. All the trichs were clear, but I trimmed, dried, and cured it anyway. It got me high, but it came on fast, then faded fast. There's more to the high than THC.