Mine is in K too. We met our friends that "clicked" through identified gifted activities. Before that it was always like you describe.
I still remember mine as a young one year old being totally freaked out and scared to death of the other children. He would say words to them and they would babble and grab at him. He was already speaking in sentences and it was frightening. He couldn't understand what was wrong with them. It was an early illustration of what was ahead.
There are gifted Meet Up groups online, university sponsored gifted programs, even Mensa groups where you can find other children he will mesh with. You might even find some here in your area who are searching for the same thing.
It is worth it to pursue because it is sad and isolating to not have friends and he (and even you) would benefit from peer relationships. (I have learned so much and found so much kinship in the mothers of my son's gifted friends.)
