Your ds sounds about the same age for grade as my dd12, who turned 7 about a month and a half into 2nd grade. She has always been young for grade b/c we started her a little early, but she's not atypically young IMO, but I also grew up where the K cut-off was 5 by Dec. 31st.
Mine is also very, very small. As a 7th grader now, she's just a hair under 4'8" and no more than 65 lbs. I also have a grade skipped dd who started school at a similar age to my dd12 and then skipped the last year of elementary. She, like others have mentioned, is taller than typical for our family so she doesn't stand out as younger.
While mine are girls and I'm sure that the issues are different for boys, a few things probably apply for both.
* Will he likely be short his entire life? Some boys just come from families where the men grow later in life and don't tend to be shorter forever. Some kids, like my dd, will be lucky to be 5'1" or so based on genetics. For a kid who is always going to be shorter than average, I figure that s/he is going to have to get used to the idea that s/he is just small and may not be competitive at sports that rely on height like volleyball and basketball. Even if the child is the oldest in grade and doesn't look as short as a result, eventually age stops being a benefit when everyone reaches their full height.
* Like others have mentioned, doing extracurricular sports with age peers might be a good alternative, but if he's in the 5th percentile of height for age, he's still probably going to be shorter than most of his age peers.
* What does your ds want? Does he mention wanting to be in the younger grade and, if so, have you talked with him about how he would feel about repeating academics?
* I can say that my dd12 was self conscious about her height in elementary. She was bothered by always being the smallest and we had some issues even up until 6th grade where her "friends" would pick her up, swing her around, and generally be too rough with her given the size discrepancy. I can say that she has reached a point in middle school now where she's pretty comfortable with who she is and her size. This came about not b/c she is no longer smaller, but b/c she has found some good friends with whom she connects on more than a superficial basis and the academic fit has become, probably for the first time in her life, pretty good. She's really starting to do better as she reaches higher level math that relies not just on rote memorization and she's had a few years to be subject accelerated there. Her classes just hit more on her strengths this year especially and I believe that she sees the benefit to being where she is academically and socially.