Sunnyday, not trying to imply my kids are bright or yours are not - my kids and I are "Not So Gifted"!
My kids were far from the brightest in their HS classes. They did a variety of activities and classes and did not attend elite schools for undergrad. They did not take classes outside of their HS. We did have some concerns that they were not being challenged, and in the case of one kid, that was probably true.
However, they turned out just fine. Or are headed in the right direction, as both older ones are back in school. I don't know that there is a right or wrong answer for your son - chances are high that he will turn out just fine no matter which path you choose.
If he wants to try for the selective school, he should go for it. However, it should be because he wants to do it. I have tried to push my kids toward certain choices at certain times, and it didn't really turn out so well.
Sorry for being touchy. I sure wish my kids weren't the brightest in their schools, that they were more of a medium fish in a medium pond at least. But it's not to be. We don't have a point of reference, we don't have peers or a source of academic humility. All we have is the kids in front of us, and our best intentions. We encourage our kids in activities where they're not naturally strong (sports and music), we remind them constantly that hard work can beat natural talent any time, and we tell them that their lack of peers will be remedied at some point along their educational journey (and that when they are in a good fit social environment, they'll see just how much their academic talent, though useful, isn't THAT unusual.) One can hope.
And yes, the kids' choices are absolutely their own; these are their lives after all! That's exactly why I am taking pains NOT to sabotage their chances at attending a selective school, just in case they want to disregard my clear preference for them not to go that direction. (Besides, I still remember being a senior and knowing that a small LAC was my best fit, but wanting to get the Ivy League acceptance anyway. I ticked that box, and the "dad's alma mater GA Tech" box, and then went on with my life. Kids will be kids!)
