Maybe you could luck into a car pool, but from what I recall, there aren't that many NJ kids in even the established private schools in suburban Philadelphia.

I understand that it is frustrating, but if you are in a high SES area in south NJ, I think there will be peers for him in the local schools. Not lots, but they are there. Some kids you can spot right away, then others, like my middle kid (HG type), don't even look like college material when they are five. Even at 17, folks are surprised when they find out the academic awards middle kid has won, or her SAT/ACT scores (which are available for all to see due to her travel team/college showcases).

My older two know kids who are globally ahead/gifted. These kids were bored in school, at least in elementary and middle school, some in HS as well. The kids who thrived on academics pursued stuff outside of school. The kids like my middle kid, who don't want to pursue any extra academics, turn out just fine too.

At this point it is confusing and frustrating, and you don't want to make the "wrong" choice. Keep exploring your options, but realize that there can be more than one "right" choice. I know I was really worked up about my youngest kid's medical issues when she was an infant/toddler. Her issues are still not completely solved, but now that things are in better place and we are almost a decade removed from the initial panic and fear, things don't seem so bleak.