Well, play it out if you said something. What would happen? Do you fear that it will escalate to physical violence? Or are you just afraid that they won't care that you said something to them?
If the latter, I don't see how that's any worse than not saying something in the first place. I mean, aren't you teaching your son now that people can walk all over you and you won't respond? How is that worse than teaching him that you'll try to defend yourself (and him!) but will fail? Is not even trying better than failing?
I'm not convinced...
Different strokes for different folks, of course. But following your logic as you have presented it here, I'm not sure that saying and doing nothing is working for you.
