Yes, it's crazy. I think when I went to HMS, 1989-1993, they accepted 10%, which was very low. I went to Northwestern U in Chicago, a very good school. However, I was the first person in 20 years from NU to go to HMS! I know Harvard undergrad only accepts, I think, like 6-8% of the applicants.
However, if you are good, you are good, and you will get in and do well somewhere. I feel very sure of that.
And I do think people underestimate how important long-term your personal skills are. At this point in my medical career, nobody cares that I went to HMS. They care that- I'm very nice, friendly, and it's easy to get your pts in to see me quickly.