I'm sorry you've been getting negative comments. It's hard when you are just trying to do the best you can for your kid. My son skipped 1st in our local school, to no one's suprise who knew him, and then we transferred him mid-year to a GT school out of town. We did this mainly because his needs were not being met, particulary pace and level of instruction. I have said things like, "he has special educational needs", "the new school works a year ahead, which my DS was ready for", "my DS already knew what they would be teaching at the local school", "we are moving to the GT school to avoid another skip", "we did not want our kid to get all As all the time; we wanted him to learn how to learn so he wouldn't struggle if he was challenged".
We have been lucky in that people who know us and our kid are happy that we found a better fit for him educationally, and we felt comfortable saying all those comments above.
If you do feel like getting into a conversation about whether GT education would benefit all kids, I'd say go for it. Some kinds of GT stuff -- enrichment, higher level thinking skills, grouping kids working at the same level -- can definitely benefit all kids. Things like doing 2 years of work in one year will not work for all kids. Also, grouping doesn't work so well when there's a group of one. A nice thing for schools to do would be to schedule math and reading at the same time for all grades, so kids could go to the level they need regardless of grade.