Like others said it doesn't matter for local gatherings. I've never came across one where nonDYS weren't welcome.
NonDYS siblings are not allowed to attend Summit, which can be a pain and does make things quite complicated. At this point DS5 is aware that his brother could attend the last 2 gatherings and he couldn't. I believe he fully expects to attend the next one. I hope he will because that means I will get to spend the entire time there as well instead of splitting the time between me and DH

For older DYS kids there can be more things like online seminars and boards available only to them.
If you only attend local actions then it's not really that important that one of your children isn't in DYS besides the "He is DYS, she isn't" which you will inevitably say at such occasions.