Well, the worms wont hurt it. If anything worm castings would improve the quality so I doubt you have anything to worry about from that. The dryness of the soil should'nt be a problem either considering that you will rehydrate it when you water your seeds. I think you'll be ok.