Gifted Issues Discussion homepage Forums Identification, Testing & Assessment Update on bizarre WJ III results

Many thanks to those of you who confirmed that I wasn't crazy to be suspicious of DS4's WJ III reading results. In case you missed it, here are the scores:

Broad Reading 138
Letter-Word Identification 171
Reading Fluency 143
Passage Comprehension 147

Brief Reading 182
Letter-Word Identification 171
Passage Comprehension 147

I pressed the psychologist for an explanation, and she said that norms do not exist for these scores at his age (4.3 when he took the test), so the system can't give him a broad reading score higher than 138.

That, of course, makes no sense to me, especially since the system managed to give him a brief reading score of 182. I really can't see how we could get nearly a 50-point difference between brief reading and broad reading, and how the overall score and percentile (99) for broad reading is lower than all of the subtest scores and percentile (99.9, 99.8, and 99.9). Besides, how could anyone ever qualify for DYS if WJ III had no norms for this level of achievement?

DS4 is easily reading (and comprehending) on a mid-second grade level, and I was really hoping this score would help him qualify for DYS. I'll keep asking our tester for a better explanation, but I don't know if we're going to get much more explanation from her.

I've heard of this happening to other people--broad reading scores that are lower than all of the subscores. If anyone else in the same boat, would you be willing to share? If there are enough of us, I wonder if we could somehow get an explanation from the publisher.

Thanks!

Thanks for the update that the bizarre situation in
http://giftedissues.davidsongifted....5393/New_to_the_forum_need_help_wit.html
hasn't got any less bizarre.

I'm sure I've seen other similar cases on this forum, but I don't know how to find them. I hope others who have had these bizarre scoring combinations can compare these cases.

We've never had a broad score calculated, but DS also took those three subtests of the WJ at the age of 4. Like your DS, my DS had high scores. I did like the high ceilings on the WJ, but if what you were told is true - the Broad Score calculation presents a real problem. It sounds like there is no point to having a Broad Score calculated for our kiddos, if they wouldn't ever reach above 138! That IS bizarre!

One thought, though, since you mentioned in the other thread about wanting to use the score to apply to DYS - perhaps submit the subtest scores to DYS as a part of a portfolio (I realize that this may not be ideal)? Could you also get something in the report from the tester stating that your DC achieved the highest possible Broad Reading score for his age?

Has anyone been able to get a higher Broad Reading Score for a four-year-old on the WJ?

Helianthus I understand how the score limits of norms are frustrating. We don't have the WJ ach scores for DS yet, but when he did the WIAT earlier this year he hit the ceiling and the Psych had to use lower scores to calculate his age and grade equivalents. It seems strange that it has to be linked and limited by age, but that's just the way it is. So it definitely seems broad score is not a very useful measure for you DS.

Since you have to be 5 to apply, will Davidson even accept achievement tests taken at 4 years old?

Just curious - sorry I can't help with why the Broad Reading # is odd.

polarbear

polarbear, on the DYS website, it says at least one qualifying test has to have been taken within 2 years. so 4 is fine, if applying between 5 and 6.

Marnie beat me to it. Yes, my understanding is that as long as the test results are within the last two years, then that's adequate for applying.

Loy, I'm not sure if another 4 year old has gotten a higher broad reading score. If so, I'd love to know--that would be a useful piece of info to share with our psychologist. I can have her put a note in his test results, but I just can't get myself to believe that 138 is as high a score as a 4 year old can get--especially when he is normed against other 4 year olds and especially if the subscores can go higher than that. If the subscores can go up to the 170's or higher, surely the overall score can top 138?

We did the testing for early kindergarten admission, and his results are plenty high enough for that. So for now I'm happy that we'll be able to look into whether early K is the best option.

So this has been puzzling me since I saw the first thread posted about it. I've been digging around in technical reports, and this is what I've come up with, in addition to (expanding on) previously-mentioned issues having to do with the minuscule number of members of the norm group at this normative extreme and age bracket:

The internal consistency reliability of the WJIII is sketchy for four-year-olds, and especially for the fluency measures, which are considered to have an adequate floor only beginning at age 7 (reading & math) and 8 (writing). While we're usually more concerned with ceilings for gifted kids, the floor does tell us that the spread at that age level is quite poor. Where the majority of kids score zero, or just above, there is not a good statistical basis for distinguishing the remaining small number of children, beyond "well above average". Consequently, the authors do not report reliability for 4 yo at all, for 9 out of the possible 22 subtests, including for half of the standard battery. The only subtests with good reliability (>.90) at this age are letter-word ID, spelling, passage comprehension, applied problems, word attack, and academic knowledge. (The remaining subtests are fair to poor.) This explains why a good Basic Reading and Brief Reading composite can be obtained, but not Broad Reading, and Math Reasoning, but not Broad Math.

Alfonso, V. C., & Flanagan, D. P. (2002). Comparative features of comprehensive achievement batteries
(Woodcock-Johnson III Assessment Service Bulletin No. 5). Itasca, IL: Riverside Publishing.
from: http://www.riverpub.com/clinical/pdf/WJIII_ASB5.pdf

Wow, aeh! Thank you for the more detailed explanation. It makes much more sense now why the fluency score could have such a strange effect on the broad reading score. I do wonder, though, how the scoring system can decide that his fluency score was 99.8th percentile--that seems to be a fairly precise measure. I know accuracy and precision aren't the same thing, but it does seem misleading to offer a precise result without also clarifying the potential lack of accuracy.

In any case, I think these results make more sense now. Thanks for your help!

You're welcome!

I think the reason subtest SSs appear to have more certain information, but composites are a bit wiggly, is probably a question of compounding unreliabilities. And yes, it would be good if a little discussion of standard error, reliability, etc. were included in test interpretations, but based on the amount of eye-glazing I get when I do venture to introduce a bit more detailed stats into my parent/teacher team meetings, I'd say that many people could just as well do without!