Something to keep in mind about kids doing ok playin with older kids (or younger) - yes it IS a theme of gifted children. It's ALSO a symptom of poor social skills in ASD kids. I have two children who do better with older children and I can clearly see the difference in the WHY between their preference for older/younger children - and the difference in quality of interaction/friendship too.
I think this is so important to think about - especially in relation to your son not having anything in common with kids in his class. My DYS son was light years ahead of his peers in preschool but he could still play with them. He preferred to play board games that he understood with adults or older children and would often craft elaborate games to play with the kids his age that they did not understand. But he was able to play with the kids appropriately even though he was way more advanced than they were. The kids in my disabled son's preschool can still play with him and enjoy him even though they are ahead of him. The kids who cannot have spectrum difficulties in social interaction. If your son cannot do this, it is still a concern for his teachers and needs to be the focus of his preschool program. Doing so in preschool is so important and can make huge differences. Later on, it is much more difficult.
While it is a theme of gifted children to enjoy older children, it is not that they cannot understand or have nothing in common with peers their own age. Certainly they can swing on the swings together and have fun even if one can add fractions and the other can't add at all. If you are teaching your son that he doesn't need to learn to play with kids who are not as advanced as he is or have nothing in common with him because of speech delays or disability, then you are part of the problem. (I believe your quote was something like "we can't blame him for not wanting to interact with these kids".) I am not trying to be harsh, just pointing out that if you want him to get over his social delays, you have to work with the teachers and help him understand that he can play with anyone even those who are intellectually inferior to him.