I'm not at the #1 program for it (turned that down because I could not do mathematics at that school), but I currently am doing an MD/PhD with a concentration in mathematics and epidemiology. My family mostly consists of people with high school degrees or less (minus my dad with a community college degree), and all of the schools that I attended before college were considered economically and academically disadvantaged.
This being said, I would strongly push for following a different path than mainstream school without acceleration or enrichment opportunities. I did not have an easy time getting to where I am now and spent a long time resenting authority and dealing with depression...
It's possible to get there, but it is not a pleasant road to travel in the least.
I have a similar experience. I "only" have undergrad degrees, but graduated w/honors from a competitive tech program that prided itself on failing out a good portion of every freshman class. I could have easily gone on to a bigger name grad program without a problem. I actually had enough credits for a grad degree with the double major.
I was never IDed as gifted and my parents were involved in my education at a pretty minimal level. They very much encouraged me to buck up and blend, and I was so intense and sensitive. I was the only person at my house who read for pleasure growing up. My parents are GT, but were encouraged even less than I was growing up. I would agree with not being pleasant in the least!
I look at my kids and don't think about big name schools. I mostly want them to have happy, well adjusted childhoods where they can be challenged and engaged appropriately. Where they can just be who they are. And hopefully they'll come out on the other end with a good sense of self and a strong work ethic.
