Aw, poor DS! I'll be watching this topic too to find out the "right" response
I generally laughingly roll my eyes and smh and say "THAT was a crazy thing for them to say then, wasn't it, because of course you can go to third grade math and obviously you're not slower than xyz's baby sister. Honestly, sometimes kids are funny."
What I'm aiming for is lightheartedness, validating my DD's situation, dismissing other kids' potentially hurtful comments without being horrible about it and making them out to be mean or stupid or psychopaths, and demonstrating how to let random comments from the uninitiated roll off your back. Don't know yet if it's working tho!
