Play is play. I don't tell other moms what my kid can or cannot do. They often share and I often tell them how great that is. I want them to feel good.
Same here. We do not talk about Mr W with most people.
But Mr W really stands out compared to his age peers and this puts off most parents of his age peers after an hour with their kids next to him.
So...it is just easier that:
Mr W prefers kids 1-3 years older than he and that is what we mostly let him play with. After a few minutes, the other parents accept what he does as that is what their kids are doing. The older kids are a bit put off for a few minutes, but accept him as a kid and not a toddler. He is not physically ready for some things like rough housing, but he tries real hard otherwise. The fact that he is still small and not as coordinated as the older kids is not as overwhelming psychologically to the parents.