He is perhaps studying from afar and learning about groups. I agree it's not a setting he seems intuitively comfortable with, based on your comments, but that doesn't mean it will always remain so.
I know your instinct is he'd refuse to participate in a pre-K program but do you have a feeling about why that is, especially if he found one or two like him? Does he think interacting with the other kids is boring? Or he doesn't know how to engage them? Do you think this would persist beyond a few rough days (I doubt any kid wants to be left out for long)? At the library, are the other kids sitting near him and interacting to his exclusion? Could you find any books you can read to him or he can read himself that model going to a new school, making new friends, group behaviors etc?
In the meantime, I agree, keep exposing him to different settings. He won't enjoy them all. No one enjoys everything. Encourage him in the ones he does enjoy, like the group swimming lessons. This may be something he likes because he can still be part of the group, but sort of practicing his own thing. Maybe a team sport like soccer?
Does he enjoy singing elsewhere? You mentioned two singing items, maybe it's just not enjoyable to him?
