I'm not a worm guy, so i'm just guessing here. But why just a couple, why not ten (or so) in each growbag?
My other thought is, would they survive the watering process? Dumping a gallon of water in the bag, relatively quickly, is a lot more vigorous than rainfall - the soil is, literally...