I don't see it as any different from teaching a kid to speak.
Something we agree on!!! But I think that children don't need to be taught to speak or to read, in most cases. In any event, I didn't do anything to help DD learn other than talking and reading to her.

Even deciding to read to her is structured, if you made a conscious decision to do it every day as part of an overall plan to help her learn.
The question is not whether children can learn to speak or read without (what I guess we should preface with "highly") structured teaching. The question is whether such teaching actually helps or not. Do you have some sort of evidence that your children would be negatively impacted if you had taught them a little more, instead of surrounding them with knowledge and leaving them to their own devices? Or do I misundsertand what you do for them?
I've met plenty of smart people with sub-par language skills as adults. Some probably post on this forum regularly.