Gifted Issues Discussion homepage Forums Identification, Testing & Assessment Large discrepancy in test results

DS is 8 yo/3rd grade. He took the WISC IV almost 2 years ago; at that time he was an 'older' kindergartner (late July birthday), and I was contemplating having him skip 1st grade and go to 2nd. He scored well - not off the charts (135 GAI), but well enough that I felt comfortable moving him up in accordance with his teacher/school's recommendation to the grade he should have been in if I had not held him. The only reason the school evaluated him was because I asked for some enrichment, spurring the conversation of 'maybe he'd be just fine being the youngest in his class' and subsequent testing/evaluation by the school (reading test, math test, a week in the 1st grade classroom at the end of K) at which point the school was confident he already knew the curriculum that would be covered in 1st grade. It was a good move to skip 1st and go to 2nd. He's the youngest and smallest in his class still, but he's fine maturity wise, and has done well academically. I was contemplating sending him to our local public school next year and he recently took the district's GATE qualifying test, the NNAT2. But he only scored a 69. I was pretty surprised. I don't believe in test prep (and he's not an anxious kid) so I told him the night before he'd take a test, told him it was multiple choice, and just fill in the scantron. After he took the test, he told me he thought it was easy/fine. After getting test results, I looked online and it turns out that are sample NNAT questions out there! I gave him about 7 questions and he got them all right (I only got 4 right!). So, I seriously wonder what is going on. Has anyone ever seen that big of a discrepancy? Or, could the WISC-IV have been that 'off'? It was administered by a highly regarded psychologist who is very experienced. I am trying to be objective but school's very easy for him, he was an early reader, etc. so his somewhat strong WISC IV seemed reasonable in that context. But the NNAT does not seem reflective. I would ask the public school to check his scantron but they'd laugh in my face...are there other factors that could cause a big discrepancy?

You didn't mention his FSIQ, which may be a stronger support if you want to appeal his disqualification. I know that some people swear by GAI but in reality he does need good working memory and processing speed for certain tests and increasingly as he progresses through school. Different tests also emphasize different strengths so both could be accurate. NNAT2 is very visually oriented (matrices) so that may not be as much of a strength? I would also not place much faith on sample questions because in my experience they are always easier (sometimes much easier) than the real test. On the other hand, trust your Mommy gut and ask them to check the scantron because I do have one high ability kiddo who could have potentially mess up a scantron at that age.

It's possible he did get off on the Scantron - I've had that happen with one of my kids and it made the difference between a typical score of 99th percentile to a 62nd percentile. However, I think it's really rare that this sort of thing happens, and my ds who had the issue has a diagnosed disability that impacts the ability to do things like fill out Scantron sheets soon... I wouldn't necessarily think that was the case with a nt kid. My gut feeling is that the difference in scores is reflective of strengths vs areas that aren't as strong for your ds. I'm not familiar with the NNAT so I can't help you with it, but hopefully aeh will stop in and help out with it (or you can also often google and get detailed info on what types of subtests are part of a test, how it is administered - group, individual, oral vs reading questions etc). Then look back at your ds' WISC report - if the GAI was used, that means there was a discrepancy in either working memory or processing speed subtests that made the FSIQ calculation not valid. There may be a clue in that info if you find a dip in either WM or PSI scores - it could suggest a relative weakness that might, in turn, have a larger influence on the NNAT (which I suspect is not as comprehensive of a test, therefore less subtests than the WISC, therefore a subtest that is weighted heavily on WM or PSI-type tasks might carry more wait in the full score).

I'd also look and see if there was any discrepancy between the VCI and PRI scores on the WISC - that also might correspond to a lower NNAT score.

Last thing - it's possible the WISC at a young age was an overestimate, and it's also possible that your ds simply wasn't performing up to his ability on the NNAT. The thing about tests is any one test all by itself without any other supporting data is just that - one piece of data. Have you looked into the requirements for admission to the public school's GATE program? The NNAT may be used as a screener to id kids who need to be tested further, but is it the only criteria for admission? If it's not - for instance, if it's given as a first-round screener and then other criteria are considered such as teacher recommendations, parent recommendations, other ability tests etc - then you can put together an appeal and advocate for placement.

Best wishes,

polarbear

Good point on the working memory and processing speed. I had to go back and dig to find the paperwork (i only remembered the highest # off the top of my head which was the GAI), but it shows:
FSIQ-128; PRI - 131; VCI - 128; WM- 113 and PSI (processing) - lowest at 109.

So definitely lower, but, again, enough to have such a low NNAT?

When DS took the WISC-IV, I literally gave him no heads up whatsoever...I did not optimize his rest, pre-test meal (none!), or otherwise make him comfy cozy. DS did complain a little bit that there was music in the background and that he had to 'turn the music off in his brain'. The doc did confirm there was background music the whole time, at a very low low level...and even offered to re-test in 6 months if DS felt distracted, but really, I didn't see the point..seemed unlikely to have a big impact, IMO, and more importantly, scores indicated to me what i needed to know- he did ok enough to suggest he would not struggle academically by skipping a grade.

I see the difference between high achiever and 'gifted' in my own kids, which is why I am questioning the NNAT in DS's case, and what could cause it to be so low. Oldest DD showed similar traits (early reader, got concepts easily, etc. though not as rapidly as DS) and had similar but slightly higher scores (139 GAI on WISC-IV and a 145 OLSAT when she applied to a magnet school at age 10..they also gave her the NNAT as an initial screener and it was 94). Her WM was higher than DS though on WISC-IV (129). My second DD is a high achiever (straight A's) but never exhibited the classic signs - not an early reader, curious but not intensely so, etc., really works to learn new concepts, etc. and I've never had her tested.

Who knows, maybe DS was 'out of sync', and a little ahead at age 6, and not now that he is 8. But he's still an avid reader (a good two hours a night); jumps on Khan academy for fun (not far ahead, just working through 4th grade math); will still work through a huge lego set alone and voraciously for hours unassisted, and has yet to have to 'review' or 'study' anything to understand it. That 69 doesn't match up with that profile, though clearly I'm not objective about the situation.

Thank you, Polar Bear. I was typing and then saw your response, and it's helpful. I'm leaning toward a private school with IB curriculum so that regardless of whether or not DS 'needs' a more intensive environment (maybe he doesn't!), I am believe it will be more interesting/challenging than what our local public school offers in the traditional classroom.

Polar Bear, in our current district, the NNAT is 'it' for gate qualification. It's all of a 30 minute test. In our former school district, NNAT was just the screener test given to all 1st graders and then again to all 5th graders; those scoring at 90 or above would get additional testing (OLSAT by district, then WISC -IV if parent wanted to provide).

The NNAT-2 is a single task, most closely resembling the Matrix Reasoning subtest from the WISC. Adding in the scantron aspect of administration, processing speed-related factors do enter, depending on the reasons for his low PSI. Even if he didn't make errors entering his responses, the additional effort of visual scanning, discrimination, and bubbling may have slowed him down, leaving him less time to reflect on and complete downstream items. It's also often administered in a group setting, which may affect children who are not as efficient at screening out background noise or visual distractions.

Was there ever any follow-up on his low PSI? Though solidly Average, it is markedly lower than his GAI index scores.

Thank you for your thoughtful response; I truly appreciate it. DS has never used a scantron before (unless he used it for WISC-IV, but has never seen one school because his school does not give standardized tests, and his teachers do not do multiple choice tests. Ever.) That might have been an issue. No, I did not follow up on his PSI (bad mom!). This may sound odd,and I'm not sure if it's because he's the youngest of three kids, but...I have always known/sensed that he would not have a hard time learning what he was expected to know. I never sounded out a word with him, yet he could read pretty fluently at age 4 picking up his oldest sister's books. I have never explained a math concept to him, and I do not know he learned to add/subtract/do basic math skills that he was not taught in K but yet knew when school tested him to make sure he knew 1st grade math. Given the 'he seems to know everything without even learning it', I did not focus in on the PSI, and I think maybe now would be a good time to look further, as he is getting older and he will undoubtedly be challenged, at some point. He attends a school that is fairly alternative and provides alot of opportunity for learning in a non-traditional sense...gardening, knitting, multiple foreign languages, mindfulness, cooking, etc. He does do reading-writing-math in school, obviously, but acceleration in primarily those areas is not the school's focus for engaging kids, and that approach has worked for him. But he has to exit the bubble, so to speak, and that PSI may come into play. Thank you again for your helpful response.

Here are my thoughts and I hope that they help you.

So, the 135 GAI should not be disregarded. Also, don't feel like you have to qualify it because it is not 'off the charts.' Think gifted or not-gifted - better accommodated in gifted classes or non-gifted classes. Someone made a specific decision to make 130 the cutoff; it has to be based on something (when I get a chance I want to research this for myself; it might be in the legislative history of the different state statutes). It may just be that represents the top 10% say for cognitive ability in any one class by grade level.

When you mentioned that specific other test I read a few sentences about it on Wikipedia, which has a criticism listed that something about that test (maybe it was new and they were working out the problems) could lead to unusual high or low scores. That is definitely worth looking into; you might help fix the problems and that can help everyone.

Then, I remembered that test might have been the one that was picked-up and discussed by the press - major magazine article (New York Times ?), radio discussion and even a small amount of television coverage. Parents were very definitely prepping their students for that test. It came up before on this forum and I remember thinking, okay, if you orchestrate your child 'passing' a gifted test, isn't anyone going to notice (especially that child) that the child isn't thriving in a gifted environment or is the parent going to force the child to work extra hard just to keep up - not a good idea for the happiness factor.

Your child is so lucky that you are aware of everything that is going on, following up is exactly what a gifted adult would do (We are constantly trying to figure everything out correctly.) Not everyone will enjoy your questioning, but hopefully they will respect it.

It is my understanding that on the "normal curve" of IQ distribution, 130 historically represents the upper limit of the 2nd standard deviation from the norm, the top 2% of the population, therefore identifies individuals with a relatively rare intellectual profile.

Yes.

With regard to the NNAT-2, I am not aware of the specific news coverage mentioned upthread, but I do know that the author has been involved in a fair amount of litigation arguing that the use of other instruments, with heavier language-loads, or more subjective criteria (e.g., teacher rating scales), unfairly discriminates against children from cultural-linguistic minorities. I mention this to give some perspective on the thinking behind its design. (I actually sat at the breakfast table with him once, while he was discussing some of his advocacy experiences with another distinguished professional in the field. Yes, I was eavesdropping!) The test is not without its flaws (as with any instrument), but it is used in GT admissions because its history is less discriminatory toward diverse populations.

In the case of the OP's child, the lack of scantron experience would probably be the place to start. If it turns out that there are visual issues, one accommodation would be to have the student mark the responses directly on the test booklet, then have a proctor transfer the responses to the scantron sheet after the test. You may also be able to have them check the original test booklet, to see if there were bubbling errors, as suggested by several PPs.

Thank you for all of the responses...good info and food for thought. I am going to ask his current school if it could give DS some type of timed test with a scantron and ask someone to watch him as he fills it out (I bet the school would do this for me). I will research a bit more on strategies to help students that may have lower processing speed. No teacher has ever mentioned him struggling to complete work in a timely manner, but sometimes, DS in his quest to finish quickly (homework, for ex), he does not do a thorough job reading instructions, etc. Thank you again!

aeh, you mention that the NNAT has a history of being more fair to diverse populations, but there is a study to suggest that english language learners scored more poorly on it than other tests. The study also found that the NNAT overidentified both extreme high and low scorers on the test. http://gcq.sagepub.com/content/52/4/275

This study was done on the original NNAT, not the NNAT2. I'm curious if the development of the new version has eliminated some of those flaws. The counselor at my son's school also mentioned that the NNAT2 was a more difficult test than the cogat, which seems strange given that they are both normed to an average of 100. His district is diverse (we live in Texas), so I'm wondering if that might have something to do with her impression, if indeed the NNAT2 underidentifies minority students in comparison to the cogat.

Sorry to derail the thread a bit, though it does point to the fact that these tests are imperfect. I would tend to trust the results of an individually administered IQ test over a 30 minute group test, even if the group test is more recent.

Appleton, I agree...again, realizing I am not objective about my own child, but that the likelihood of results from of an individually administered comprehensive test being completely 'off' is less then the likelihood of results from a 30 minute group administered test being completely accurate:). He took the NNAT2. Honesty, I don't think our district knows 1/100th of what the folks on this board do about tests. For example, when registering DS for the district's testing, the coordinator only referred to it as the 'GATE test'. I finally asked 'what test will he take'? and the response was 'it's the GATE test'...I responded that there were a number of tests that could be used (NNAT, OLSAT, etc.) and all I wanted to know was when to drop him off and pick him up and some tests are longer than others...geesh!

A drop of basically 4 standard deviations (assuming sigma=15) just does not seem plausible without something else going on. This isn't regression to the mean as much a 'reflection' about the mean almost as though someone was reading from the wrong end of the chart. I would follow this up IIWY

This raises a good point. One tricky thing about computer-read forms... within each section, some are numbered horizontally (across in rows) while other computer-read forms may be numbered vertically (downward in columns). Therefore it can be easy to mismark if one is focused on the questions and not being mindful about the answer sheet. Stray marks and incomplete erasures can also be problematic.

I think the OP means that her son scored in the 69th percentile,not a 69 SAS. The SAS for 69th percentile is somewhere between 105-110, which is around 2 SD different from the GAI of 135.

Good point, I just realised this and just logged on to delete my post but was too late - whoops

First, of course, individually-administered tests are better measures of cognitive ability for pretty much anyone. They are more nuanced, cover a more diverse range of ability-types, are more adaptive, and allow for querying when children give atypical responses.

Second, as to the study you linked, that study compared the NNAT to two other instruments administered as group cognitive assessments. As it happens, it was authored by the author of one of those tests (coincidentally, the test that the study found to be the "best" was the one by the author). Not knocking the research, but giving some perspective. The Ravens, which was found not to be properly centered (the mean is off), is a very old test, so that's not a surprising finding. The CogAT has a much larger standardization sample than the NNAT, so it is also not surprising that it would have more accurate norms. Note that the article is about comparing three nonverbal assessments. We can only see the abstract before the paywall, so I don't know what's in the rest of the article, but the title suggests that the comparisons were between the CogAT6 Nonverbal and the other two tests. This is not how schools commonly use the CogAT.

This study comparing the CogAT6 to the NNAT-2 found that the NNAT-2 had slightly less of a gap for Hispanics and ELLs than the CogAT Composite, with gaps significantly affecting GATE identification for Asians and ELLs. The CogAT Nonverbal was slightly better than the NNAT-2. They also found that follow-up WISCIVs matched the Composite scores the best, which also makes sense, since the domain overlap should be better: http://gcq.sagepub.com/content/57/2/101.abstract

The history of schools attempting to use figural tests for GATE screening reflects lawsuits in California back in the 80s and 90s regarding overrepresentation in special education of CLD/minority populations. Courts decided that the face cultural load of verbal IQ tests discriminated against diverse populations. This was the impetus for research and design into updated nonverbal tests. Item analysis since then has found that there is actually negligible cultural bias in most of the gold-standard cognitive assessments (though, naturally, LEP factors continue to affect performance on verbal tasks). So yes, it is quite possible that the CogAT6 composite does not discriminate against diverse populations any more than nonverbal instruments of equal psychometric robustness. One can make an argument that nonverbal assessments discriminate against verbally-skewed students of all CL backgrounds.

However, to my knowledge, the history of GATE screening with and without nonverbal instruments appears to support their use as part of the process, in terms of the actual shift in diversity of students accepted to programs. Whether use of these instruments continues to be necessary to level the playing field for CLDs is an open question, especially as newer tests come out, and CLD factors are given more attention in test design and item tryout.

Bottom line, no one score can be used to describe a whole person. Obvious, right?

Sorry, especially slow today, what does CLD stand for?

Appleton - yes, sorry, I believe the administrator meant 69 percentile because I asked 'do you mean percentile'. AEH, thank you for the additional information. I was aware that the NNAT2 was a preferred nonverbal assessment to level the playing field but the research you linked is helpful. We are in CA though we have among the least diverse districts in the state (less than 4% ELL, less than 10% minority in total). I assume our district also uses NNAT2 because it is a short test, but that's a guess...the district gives no information on its methodology in GATE testing process, because, well, why would parents need know anyway:)? Interestingly our old district gave the NNAT2 to all 1st graders, and here, it's 3rd grade...

catova, do you have a copy of the test results or were you just given the info verbally? If you don't have a full copy of the test report, I'd request it.

polarbear

No test results were provided; just an email with score saying DS will not qualify for GATE services. Coincidentally, I just requested a copy of results. We'll see what the response is.

Oops, is my jargon showing?
...CLD means culturally/linguistically different (or diverse).

Thanks for the info aeh, interesting study. Of course now there is a form 7 of the cogat so who knows how it would compare. It does make sense that it would correlate more highly with the wisc since it measures more domains than the NNAT.

I did notice that the study I mentioned was written by the author of the cogat, which is important to mention. In his newsletter cognitively speaking, there are some bar graphs of stanine scores by different groupings, which I thought were particularly interesting. I always assumed that testing would follow a nice bell shaped curve and obviously that's not always the case. Granted these were not representative studies, so who knows if the original norming of the instruments were a bit off, or if the results just reflected the characteristics of the population being studied.

Catova, when my son took the test, the school gave us an NNAT2 "home report". It lists the "Naglieri ability index" and the percentile rank. There is also a nice bell curve with descriptors "average", "high average", "superior", "very superior" etc., and numbers below with a dot on the curve to indicate the score. There is no raw score or number attempted listed. His school had them take the test on the computer, so it's probably easier for them to print out that info for the parents.

I don't know if this is helpful or not, but my DS10 took the NNAT and scored 83. He was privately tested with the WISC two weeks before and scored as HG but likely twice exceptional. At the time I didn't know his second e, only that he had terrible anxiety and slow processing speed according to the private tester, however the private tester felt his scores were high enough that he would be fine on the school test. Because of all this, when I got his score of 83 percentile I called the GATE office and told them I didn't believe those results based on the fact that I'd had him privately tested and he was gifted- they went back and rescored his results and they went up to 91st percentile. Still not qualifying but higher. Perhaps because it was a group test, that although he said he enjoyed the test he was distracted, maybe bubbled something wrong (which maybe is what they caught when they rescored) In addition, aeh weighed in and said based on his scores on the matrices on the WISC his score wasn't too far below what could be expected (he scored higher on the verbal parts of the test). So he needed to be tested with a different instrument, and with an individual test. He also needed help with his second e(s). However, and this is important- I do not see giftedness in him most of the time, at least nothing like what I read about on these boards. He is extremely creative, and does very well at school, but not above grade which is what you would expect to see for someone with his GAI. In your child you are seeing the giftedness, so I would trust that over the test. Aeh is right about the test, it is not the be all and end all - it is a snapshot, and to continue the analogy, using a particular type of camera. Sometimes the photographer isn't very good. Sometimes you aren't feeling particularly photogenic. Sometimes the camera crops you poorly. It doesn't mean you aren't beautiful

Also, call the GATE office (not your school) and request the raw scores in writing (I faxed my request and got the results less than a week later). Even if it doesn't tell you that much, I keep all testing for my kids because you don't know whether or not it may be helpful later. Also if you can, voice your concern to the relevant person at the GATE office and ask them if they would be willing to look at the scores one more time based on his previous scores and if your teacher will back you on that even better. And be very very nice to everyone you speak to in the GATE office, it helps. When I posted about this before on these boards, I was told that someone had actually received another child's test results. Mistakes happen, if you think something is off, be very nice but confused about how there can be such a great discrepancy and call the GATE office and ask them what they think. Again, if you speak to the teacher and she is also surprised by the score, tell the GATE office that as well.

*Edited to add that I just read that you requested his scores- I did that initially but only got his percentile- make sure to request RAW scores. They will not send them if you do not request them specifically.

And once you have sorted this test your untested child. It is not always obvious and while it is possible s/he will score average they also may score HG or be 2e with the second e concealing the first through compensation.