My DD (already young for grade but about 50th percentile height-for-age) cared very much about not being the shortest kid in the class when we were planning her skip. But it was important to me because she cared about it - I was always one of the shorter kids, and never minded it.
Social immaturity would worry me more, but forgoing a skip doesn't always help with that. The oldest kid in DD's class last year was a very bright kid who had been held back purely for social reasons. Before I knew that, I'd figured him as either young for grade or skipped, because he was still socially immature compared to his (year younger) classmates.
FWIW, my DD's social situation appears to have improved after the skip. She's not the type to have a lot of friends, but the friends she has now are bright kids who are well-liked. (Prior to the skip, her friends tended to be academically struggling kids who didn't have many other friends.)
