Labels for kids like your son are just words that we stick on a cluster of symptoms. If the label can help him to get accommodations that will make his life easier, then go with it. But if you don't feel that the label applies, then don't think of it as written in stone.
My oldest has AS but has chosen not to inform his college. Therefore, he gets a lot of negative attention for his quirks, and no support. A professor wrote on his semester eval that my son would only speak up in class when he got angry, and then he'd make some very interesting points. He didn't get outside of his little box though, unless he had very strong feelings on the subject. I won't bore you with the difficulty he has had in getting a job, but it has been a real challenge. It doesn't matter how smart you are if you can't relate to people.
I just got Jenny McCarthy's new book, Healing and Preventing Autism. I'll let you know what I think after I read it!
Hang in there-