Perhaps his family is voicing opinions on the program? Not sure where they stand on the gifted program from your post.
Here is an argument that might work for you, given his 'segregation' argument. I, personally agree with the argument and frustration of the way most schools organize their education programs. There is something to be said about the old school house method and teaching mixed age and abilities together. BUT we don't control their organization. If they offered mixed abilities that would teach to everyone's level not just the general range then I would be all for it. Right now the only choice you have that would allow your son some opportunity to learn (which is what we want for all children: a chance/right to learn) is the program that 'segregates'. And how about the Special Ed students from the other range? Are they segregated? IF they are you could use that in your argument. I know a lot of school districts in our area are attempting to not segregate the Special Ed groups but your district might still do it.
Here is the other question ... if he is anti-gifted for your son because of the reasons you provided; does this also mean your daughter will need to be pulled from the program? You might want to remind him of later issues with your children if you the parents made the choice to allow her into the program but not him. He might grow to resent you.
