I can only imagine how depressing this must be. frown To start, don't assume that the school or the skip have anything to do with it. He's in a very different situation now and he may have had the same issues as a 4th grader at the public school as at the private school. Frankly, I think 4th or 5th graders can get a lot meaner than second graders (unfortunately!). My dd was young (for grade to begin with), small and grade-skipped. She had a rough time socially early on but I really think it had a lot to do with her. She just didn't "assimilate" well. I honestly don't think she was trying to show off, but others took it that way. I think part of the issue, too, was that she got more inept/awkward socially as she got older, and kids got meaner. Sigh... (It's good now, as an adult and out of college, but I think she works at it).
The biggest question I had when reading your post was why you hadn't heard this from the regular teacher/principal/social worker. No matter whether he stays or goes, it sounds like social stuff doesn't come easily for him and that needs to be a focus. When our dd was diagnosed as PG (while being evaluated for anxiety and other issues) the doc told us very firmly that we'd have the rest of her life to work on the academics, she would always be this smart, but working on the social/emotional was really important. I'd consider an outside therapist as well.
I don't think I'd move him back a year. There were times when whatever was wrong with dd (common cold ;)) certain teachers blamed it on her being young. I honestly don't think so - I think the combination of her being so gifted and a little off-the-norm socially would have been an issue no matter what her grade. Granted, it's probably easier for girls - she was petite and very pretty and didn't really have to worry about sports outside of dance, iceskating, cheerleading, etc...But I would also worry about your ds's self esteem. Dd always felt a bit different but amazingly she really didn't believe how smart she was (or at times, and this didn't help socially, that other kids really struggled and not just due to lack of effort).
I would definitely start with a meeting with the regular teacher. I know the mom/sub was really clumsy, but I'd like to think she *thought* she was helping you. Sometimes you need to just say thanks for your opinion and let it go...
Good luck, hang in there and please, please don't assume that anything YOU did or planned caused this. It might be much worse if he was still completely bored academically and struggling socially as well!