Homeschooling can be great! Have you looked into homeschool enrichment programs in your area? Where I live there are tons of choices for both public and private programs- anywhere from a half day a week to 3 full days a week. Some of them cover core subjects and some do the arts or classes like brainstorming. You could handle the subjects he needs the most acceleration with at home, and all the things you'd be worried about him missing could be covered through an enrichment program.

And if you look into it and your area doesn't have options like these... why not get the ball rolling to start one yourself? Homeschooling is gaining popularity, and chances are there's more than one family nearby who's in your shoes.