We are in a very similar situation with DS (in 2nd grade). Except his "walking around" eventually led to more disruption and trips to the principal's office. He has done less and less homework/schoolwork as the year has progressed, and now pretty much refuses to do anything related to the school unless he's in a good mood and it seems interesting. It has gotten to the point where he is angry at having to be at school much of the time, and transfers that anger to me for bringing him there.
The result? We're on the verge of pulling him out of school, because it seems to be doing more damage than good. DS has some SPD issues that have gotten worse in the last couple of months, and I do wonder if this is in part to him being so frustrated with school.
I would be wary of the school's stance of buckling down and doing it. My kid can buckle down and do something boring once in awhile, but asking him to do more or less meaningless work day in and day out has been hurtful.