I admit that now my nine year old is beyond that...but at least he does like Harry Potter and that gives him a subject to talk about. He can practically quote the books verbatim, but that's another story...
Are you sure you're not talking about MY DS9? *lol*
No real advice for the OP, but we have been in your shoes, so maybe our experience combined with those of others will help ease your mind.
Until DS was partway through 1st grade, each time we'd visit the school at recess, he'd be he'd be walking alone around the perimeter of the playground. It made me want to cry. Some of it was obviously a battle with perfectionism in some of the outside games. He was horrified to miss a basket or get called out in kickball in front of everyone. When he did play with others, it was primarily tag-type games. Thankfully, his self-esteem and self-confidence seem to have blossomed in the past couple years.
The other piece is I think for these kids who are so intellectually advanced early on, the gap really is HUGE with the other agemates, and it's difficult for the child to understand especially if they're used to being around adults. While I will not say the unthinkable "everyone evens out by 3rd grade"
I do think that the gap becomes less a bit less glaring, especially as some GT kids will learn how to speak to their agemates in a way that is accepted.
DS9 in 3rd grade now has 2 pretty good friends. He also seems to be comfortable that he's not a "trendy" kid. DS still doesn't like many things that are popular, and the things he loves are generally not mainstream, and he's OK with that. Bottom line, is DS the most-popular kid in class? Not by a long shot! But he's pretty happy these days.
I think it's definitely worth it to try to find an activity that your DS likes where he might be able to connect with the others kids. If it doesn't work, I'd be willing to bet that in the next couple years, the situation will improve some. As DC have gotten older, it's become a bit easier to find activities (e.g., chess camp) where there are frequently other kids that they relate to.