One thing I'd be careful to do in this situation is make sure your child knows that you hear him, and you're taking his concerns very seriously. If you're opening a dialog with the school about what accommodations can be made, make sure he knows you're doing it.
The reason I say this is because my DD made the same comments in 1st grade, and her attitude was going downhill at home. It turns out she thought we were the ones at fault for placing her in 1st grade. We thought we'd already made her aware that by that time we'd had a few meetings with the school where we were adamantly arguing for a grade skip... and getting nowhere. They were offering instead some accommodations that just weren't making the grade (and still aren't).
Finding out that her parents were in her corner did a lot to relieve her anxiety about school. Otherwise, she's sitting there miserably in school and thinking about how none of the adults understand and/or care about what a bad time she's having, so she can't trust or depend on any of them. Now at least she knows there are two she can.
