Wow, thank you everyone for your replies!!! It is a difficult issue for me to think through because there are so many emotions clouding my "gut feeling" -- sadness that he's growing up so fast, missing our time at home together (kindergarten is 6 hours per day, twice as long as preK), admiration for how advanced he is, and worry over every little thing that could possibly go wrong!
I got a better look at all his classmates, and he's actually not the shortest. He's 50th percentile for both height and weight for his age, so I guess he would fit in ok with either the older or younger group. He really is a pretty mature kid as far as sitting still, taking turns, etc. and his fine motor is ok too. And he seems to be starting to make some friends in the class!
What about letting him do private K this year (he's already in the class, right?) and then seeing whether you want him to repeat K in the public school next year or whether you want him to go to public 1st. Then you'd have a whole other year to gather data and observe.
I guess this is what we'll do for now. Much as I miss him, I couldn't keep him home now that he's experienced the fun of school. He complains about having to leave his toys in the morning, but after school he always tells me that he had a great time. Even if I did keep him home, I couldn't stop him from continuing to learn. Might as well give him a chance to hone those social skills and get used to doing homework (which seems to be going better this week than last week!).
I also appreciate the suggestion to visit the public elementary school and learn more about the GT program there. Keeping him at the private school is an option too, although we'd have to adjust our financial plans.
As for sports, he's not very athletic right now but we hope he'll develop some interest eventually, for fitness and social reasons. My husband loved swimming and wrestling in HS even though he wasn't the best, and maybe those will be good choices for DS as they're less "team"-oriented.