I agree with Nautigal.

It's your DH's family, and conventional wisdom says he needs to deal with it. But in your place, I'd make it plain that if nothing changes, I'm going to try to keep DS away from them as much as reasonably possible, at least for a little while, to give the grown-ups some time to grow up.