My Bear(2.5yrs) works along the same lines as the other kids mentioned here. He has a very low frustration level and some sensory issues as well as having trouble separating and being amazingly timid. He does well at the playground, but any family events or outings where there is chaos or lots of people or a bunch of noise and he falls apart. He also has an incredible vocabulary, but poor pronunciation and tends to repeat himself a lot.
What I did to try and get him comfortable with groups and a certain amount of chaos was to enroll him in Music Together classes, similar to the Music For You classes mentioned previously. The ages range from 10 months to 5 years. Basically there are NO requirements for the kids other than to be there and not wear shoes on the carpet (but Bear got a pass on that one because he simply couldn't function without them... if they were off it meant he couldn't leave. Sigh...). Anyway, he spends the whole class glued to my lap, doesn't interact with the other kids, doesn't talk to anyone, and doesn't do too much of the music part except for the parts that involve instruments. We have to go sit outside of the group when they do the dance parts because he just can't deal with all the movement and the quick transition from sitting to dancing, but we normally ease into it by the end of the song. The important part is that he really loves the classes and is slowly (painfully slowly) starting to involve himself. He's been getting us instruments and helping to put them away. Recently he even joined in with a couple other kids to push the instrument bin back in place.
I think the simple fact that there are NO expectations placed on him to participate or perform is what is allowing him to relax enough to start joining in. That's probably the same reason that the playground is easier for him than other venues even though it is chaotic.
If you could find a place where your little guy could interact with a group where there are no expectations on him at all, not even the expectation to interact if he's not ready, might help in time.