Yes, I think that the format change is part of it. The counselor told me that the k-2 version is picture based and untimed vs. a timed test with words and numbers for third onwards. She also told me that the scores are not really supposed to change that much, but that they often do. The non-verbal part of the test is the section I would expect to change the least though (it would still be pictures), and that's where his score jumped the most. Maybe it's all the hours of minecraft that he plays
I think the first grade scores are probably the most invalid - it's the first test he had ever taken on the computer and I suspect he just wasn't mature enough at that point to really try his best. I also did not explain to him why he was taking the test that year which I think effects effort. He's also a fast finisher, so I suspect he got distracted while waiting for the other kids to finish so that the test administrator could move on to the next question.
Regarding a practice effect - I think that's possible too, though a friend who also has taken it three consecutive years had her scores go down compared to last year.
My son is obviously stronger in math than language arts, so the test is picking up a valid difference there. I don't know what to think about his non-verbal abilities though - that's not something I'm really tuned into and doesn't match up with a specific school subject.
I find it all very interesting though and wish that the makers of the test would study score reliability over time.
