I'm thinking that, if you are enjoying it and your DS is enjoying it, continue doing it your way. The other moms are out of line and if their kids are "getting distracted" by his creativity and talking, then that's really their issue, not yours.
If you're not really enjoying the activities but still want him to learn to do things that teach him to follow directions or sit still for progressively longer periods of time, then there are a lot of things you can do. You can talk walks and allow him to stop and draw or photograph specific things, and then perhaps do artwork with things he finds. You can have a friend or two over and have your own storytime, ideally switching off with other mom friends leading the group. Bake. Have picnics. Go to outdoor concerts where sitting is encouraged but wandering is allowed. Build stuff in the backyard. Garden. You get my drift . . . .
Of course, if homeschooling were something you were considering, perhaps creativity and talking is more important than learning to sit still and follow rigid art rules as either a toddler or preschooler.
To me though, you're doing exactly what your DS needs and it's the moms at your activities who are being unreasonable.
