I do a lot of this double-thinking as well. I think that there are many of us gifted kids who were bored in school, but found plenty to keep us entertained. We were smart and interested and used the spare time we had from not being challenged to learn all sorts of things that interested us or participate in things we would maybe not have otherwise done (I, for example, was in charge of fruit cake sales for the honor society and took tap dancing lessons!). Then when we got to college, it was like a whole new world opened to us and we were very happy. I, frankly, don't think DH or I were really harmed from our lack of challenge. We went to top ranked graduate schools and are thriving in our fields.
But I know pleny of people who were harmed--they became frustrated and angry at school and eventually the world. They felt completely mis-understood. Because they couldn't concentrate in school, they actually decided they weren't smart. They dropped out and continue to carry the scars with them well into adulthood. In the right atomosphere I think many of these people would have thrived and gone on to be at least happier if not more productive. This waste makes me very sad and angry myself.
We each need to look at our kid and our unique situation and decide what is best. I have one of the happiest, most resiliant kids you can imagine. He is an extrovert who can advocate for himself really well. And I have school that recognizes DS's talents and doesn't balk at subject acceleration. The psychologist who tested him a few years ago said that we had about the happiest kid he had ever seen. He said, because he is gifted, he is at risk for getting bored and turning off to school, especially without any grade skips. So he asked us to watch him carefully for any signs of problems. He said for a kid whose baseline was so happy, it wouldn't take much of a change in attitude to signal there was a problem. So we watch and wait. We haven't seen problems and we have not made many changes, but I keep an eye on this board and see what people are doing because I want to be ready to act swiftly if there is a problem. I don't want to lose my happy kid to anger and despair--it would be a huge waste! And many of the parents on this board can tell you that once the damage is done, it's hard to undo.