I think we all need to find a balance between the needs and requirements of every family member.
I am a parent of a gifted 3rd grader in his 4th school, and we moved schools because of poor fit. As my child grew, his needs changed a lot and what was a good fit for preK was not a good fit for K and so on. My child had close friendships in all of his previous schools and was reluctant to leave one of his schools because he thought that he had worked very hard to fit into the popular group there and delayed the school change by several months. In hindsight, he tells me that his new school was a much better fit and that I should have put my foot down and pulled him out of the old school long ago!
Like Mom2Two, my DS has very strong social skills and has no problems making friends wherever he goes and he strikes a good rapport with his teachers in all his new schools. It seems like he is being extra social to fit in quicker. Moving also gave him a new school setting where for the first time he is in a group with multiple PG kids and he is challenged in a new way because he is no longer the only kid who had all the answers and he is listening to opinions and view points that are very thoughtful for the first time in his school life.
So, don't worry, your son will find a way to fit in anywhere you go and there will be more friendships to look forward to.
