I'm a first timer with clay balls, so my advice is limited but here's what I did:
I measured out the amount of balls I needed rather than trying to wash a whole sack full.
I washed them out in net baskets under a tap untill the water ran clean(ish), and allowed them to dry for maybe 24hrs.
I then did that 2 times more, at which point I put them all into a bucket of water and gave them a really good swirl and fished the balls out to dry.
I decided the amount of sediment left in the bucket was negligable.
The main issue, IMO, is that when brand new they can be very dusty, so it's really just a case of washing the bulk of the dust out so your run-off water is clear rather than brown.
If it's too dusty it could cause issues with pumps and filters etc, and also potentially mess with PH balance, but realisticaly you'd be washing them forever to get them 100% dust free, so it's more a judgement call to get them pretty clean.