Our DD is 3 and an only child -- she exhibits similar social "quirks." I get exhausted dealing with people who claim there's something wrong with her because she doesn't like children her own age. She gets along with older children and adults, but does better with small numbers of people and with people who will speak to her and respect her boundaries (physical space, in particular). Seems to me she's just an introvert with strong personal space boundaries -- nothing wrong with that, but not a typical 3 year old behavior.
She isn't in preschool yet, and I'm anticipating some of those same issues when she does start. I would trust your instincts. If he's warm and loving with other people (just not his classmates), it seems to me it's more of a classmate/school issue than something about him that needs to be "fixed."
