I'm so sorry you feel so discouraged and I completely understand your frustration. My thoughts -
DS started reading at 2.5. At home, we are reading chapter books such as Magic Tree House (not sure what the level is, certainly not MAJORLY advanced, but pretty sure it's above a D). I asked how they came to this level, and the teacher (without any actual assessment in hand) explained that it probably wasn't his fluency but his reading comprehension that was struggling.
Several thoughts on this - my dss also were assessed WAY below level in K. Older ds went from a level D to an S in kindy, which of course, only "confirmed" for the school what a great job they were doing (not that they had completely missed his skills on the first test.... GRRRRR!!). What helped for me was sitting down with the teacher and looking at his evaluation sheets. I found that they wanted very specific answers and often, my ds (the one with speech issues which eventually were diagnosed as including expressive language issues)simply didn't say enough. When *I* talked to him about the stories, it was clear he understood everything. I had to ask the teacher, not to prompt him exactly, but to say, "and???....." He didn't need hints; he just needed to know that they were looking for more. The grading system is really weird. To give you additional commiseration, he was tested in 3rd grade at an "R" level - apparently his reading got worse from K to 3rd?? Yeah, right.

My ds' levels were everywhere: 99% in certain subtests and 10% in another. In fact, that one was letter identification, which everyone agreed meant that ds was clearly CLEARLY not invested in the assessment process. So essentially, they gave me all this information that, not only was ds not ahead, but he was potentially barely meeting standards. WTF?
My ds had some of the same issues. He started reading very early and by kindy was reading the first Harry Potter. BUT - he hadn't necessarily learned letter names or sounds. He wasn't interested in the phonics aspect because he was so far beyond that. Also, he had been a spontaneous reader, not one who was taught or who learned by separating out sounds, etc. The AIMSweb things were easy for my second son, because even though he was also a fluent reader, he had asked to be taught to read in preK and had followed the traditional "this is B, B makes the bbbbb sound." So, your son may not even be "not invested," he just may be so beyond that level that he really hasn't thought about breaking down the words into their component elements.
I get the school can only see what ds shows them. But, then the teacher explained that she had given ds a challenge packet to work on (optionally) and he has chosen not to do so
Wow, I feel like we have secret twins wink Okay, my son had the same issue. One, he is really smart but wasn't particularly quick/good at OT in kindy. So, the challenge packet *after* he did everything else was useless. Secondly, it was still the same boring, mundane, busy work that was in the other work, just harder. He preferred to read in his free time.
I asked my dh why he didn't speak up - but he said it wouldn't mean anything if ds isn't showing him his abilities. It would've meant something to me: that I am not crazy in my assessment that ds could've passed K curriculum LAST year (or even earlier).
Try not to let this come between you and dh. My dh is fairly quiet, factual, unemotional. His typical response to me re: these meetings was always, "what is your objective in telling them that?" I wanted to vent my frustration at the school, but he only believed in saying something with a specific goal in mind and one that he thought would be achieveable. If they weren't going to agree, he wasn't going to waste time arguing. I found over many years of IEP meetings, I needed to very specifically tell him in advance, "I need you to back me up on this. I need you to tell them that our child can do this." It wasn't that he didn't believe it, or agree with me, but he never talks without a specific, achievable purpose in mind.

All in all, I totally get your frustration. I was where you are at in kindy, and even worse, ds ended up tanking on his first OLSATs (we realized later that oral directions were hard for him to process... when he took them the second time a year later-with written directions-he went from 18%-ile in nonverbal to 99th%-ile). For the most part, I just kept repeating myself, and really trying to work with the teacher on the assessment front. When we had conferences (at my request), I'd often have ds there and I'd lead him through some stuff. I'm not trying to make this sound like a dog-and-pony show, but for example, anyone reading your story about positive and negative numbers should GET that your son is atypical. But, the average assessments in kindy just don't allow your son to show that. Eventually, the teachers caught on. Now, he's in 7th grade, in a great gifted program, rocks his testing, rocked the SAT last year, all good. With ds10, I think I had a better handle on it and pushed harder for enrichment (thus he now goes to the MS for math). It does get better, but it's very frustrating at the time.
Hang in there and continue to advocate. You're not crazy, you're not off the mark, your son is NOT a typical kindy as far as academics.
Good luck!!