I think any time you're going off the "standard" route for any reason there will be fallout. Grade skipping? They've met someone who grade skipped and never recovered from the trauma. Homeschooling? They have a weird cousin who homeschooled and never learned to read. Taking the SAT young? or learning Latin? or paying for music lessons? or entering the science fair (for Pete's sake!)? They're pretty sure it's all the parents doing the work, living through their kids and/or hothousing.
Basically anytime you stick your neck out someone will take a whack at it. (And try being the grade skipping homeschooler whose DS won the science fair last year! LOL) So if it helps, keep in mind that there are plenty of people willing to complain about anything, with absolutely no correlation to the validity of the complaint. You could be perfectly average with no aspirations or interest in anything and someone would find something to complain about. So you might as well go ahead and do what's right for your kid. Some of them might be convinced by counter-examples, or explanations of the potential downsides to NOT making a change, but when you find one that just holds on to their position for dear life and won't listen to logic, it's not worth the effort to argue. (Still tempting though.... lol...)
