I think one thing to keep in mind is that many folks will never come into close contact with a HG/PG child. They will, however, probably meet several 'smart' kids. It's pretty hard, I think, for someone to really understand a HG/PG child if they've never been exposed to one, it's easier to think the parents have inflated opinions of their child.
This is one of the main reasons DS8 is taking the WJIII. So that we have something more concrete than everyone just agreeing that "he's so smart!" and not wanting to do anything about it.