Hi KsK,
I'm with Dottie that all bets are off, and I will admit that 'doesn't like to do easy things' can be classic 'smart plus ADHD' behavior
but the question in my mind becomes, how to reduce the anxiety. When does he seem happy? When does he seem more stressed? Do you think your son is 'ashamed' when people ask him to do easy jobs that are way, way below his ability level. (This was my son's case, although mine was still able to give the testing scenario a chance, since it wasn't the continual disappointment that school was.)
What is your son like as a reader? Does he have favorite books? What are they? Does he talk about them in a way that convinces you he gets them? Does he talk about them with age peers?
Does he have friends? Does he seem to enjoy them? Is he being bullied at school? Made fun of or dismissed by the teacher. I would get my butt into that classroom to observe for a few visits, for about an hour at a time, even if your work schedule is crazy - I would want to see for myself what is going on.
I'd also invite you to consider 'afterschooling' with your son as a way to get to know him as a learner. Watching him tackle challenging material is going to tell you a lot about him.
One thing we know is that he is quick-quick-quick. See how that plays out while he is learning, and compare it to what you saw in the classroom. He may not be gifted by your school's local definition, but I give you permission, based on those scores alone, to say that he is gifted in the areas of problem solving, and perhaps bottlenecked in his ability to express (or verbally absorb) ideas.
(You don't necessarily count, because he is at his best with you. Can he communicate with folks who aren't already in love with him? For an anxious kid, trust is going to be a huge issue, and for some kids it's 'I don't care what you know until I know that you care. BTW, how does he do when you are telling him about a book your read, a movie you saw or a childhood experience?)
If you can't do the 1-1 with him, I would find someone else (outside of school) to do it - I get the feeling that he really needs someone to focus on him as a learner right now.
Love and More Love (and I mean it!)
Grinity