Gifted Issues Discussion homepage Forums Identification, Testing & Assessment Group ability tests vs. IQ tests

I am starting a new thread on this topic following the topic having come up on a number of threads including this one: http://giftedissues.davidsongifted.org/BB/ubbthreads.php/topics/157401.html#Post157401

A few posters here have mentioned actual studies that show the CogAT to be a poor predictor of giftedness, but no one has posted any actual studies to my knowledge. The only one I had seen was the old one from the 1980s that I posted in the above referenced thread on the OLSAT vs. the WISC-R.

I have found a few others though, although they are not available in their entirety online as far as I can tell. Here's what I've got thus far:

An Examination of the Relationship Between the WISC-III and the Cognitive Abilities Test for Selection of Students for a Gifted Program - this appears to be an 84 page dissertation with the following excerpt/abstract:

http://books.google.com/books/about/An_Examination_of_the_Relationship_Betwe.html?id=mX5cOAAACAAJ

The Woodcock-Johnson III and the Cognitive Abilities Test (Form 6): A concurrent validity study - pdf available in its entirety online here: http://faculty.education.uiowa.edu/docs/dlohman/CogAT_WJIII_final_2col-2r.pdf

with the following abstract,

Thank you for doing this!

I may be the only one contributing to this thread - lol! Feel free to add, though, if you've got anything.

Here are some more studies, though:

A Comparison of WISC-III and OLSAT-6 for the Identification of Gifted Students (http://cjs.sagepub.com/content/11/2/120.short)


Identifying Academically Gifted English-Language Learners Using Nonverbal Tests: A Comparison of the Raven, NNAT, and CogAT (http://gcq.sagepub.com/content/52/4/275.short) - this may not apply to many of our kids, but is interesting none the less.

I have nothing. But I have to get my ducks in a row for dd6. I anticipate this being an issue.

To be fair, I do believe that there are a number of studies that show that the verbal and quantitative scales on the CogAT are good predictors of achievement in those domains and the nonverbal is less of a predictor of achievement. I'll edit to post links to some later.

Here's my hypothesis, though: I believe that the verbal and quantitative measures on the CogAT (the group ability test with which I am most familiar, b/c it is used extensively locally to me), are measuring exposure, knowledge, and achievement in those domains rather than ability in those domains. The non-verbal part, OTOH, which tends to be less considered by schools in determining giftedness, I believe measures visual spatial and abstract reasoning abilities more than achievement and also correlates less with achievement b/c it is more indicative of a divergent thinker not a child who can learn and regurgitate standard material easily. I'd tend to think that the non-verbal part of the CogAT is a better indication of giftedness as I define it which includes ability to create new knowledge and see connections and creative solutions more than repeat back knowledge that someone else has created.

Not really a study, but an author who tends to agree:

Increasing Minority Children’s Participation in Gifted Classes Using the NNAT: A Response to Lohman (http://gcq.sagepub.com/content/49/1/29.short) - keep in mind that this guy is the author of the NNAT as Lohman, who wrote the original critique of the NNAT as compared to the CogAT (above) is the author of the CogAT.
emphasis mine

This gets interesting here b/c Lohman, the author of the CogAT, has written extensively on his belief that ability and achievement are not separate constructs. I have to admit that I tend to disagree in that I've seen high ability people who are not high achievers and the reverse. I wonder if his thought process leads toward beliefs more in line with the current "talent development" model that the NAGC, for instance, embraces which requires high achievement and motivation to be considered "gifted."

For instance, here's a newsletter on the CogAT in which Lohman writes,


and

http://155.44.225.10/products/group/cogat6/pdfs/newsletters/CS_vol4_summer05.pdf

I do agree with him in some areas, for instance,

however I am not ready to say that we just call high achievement and high intelligence the same thing.

Thoughts?

eta: to drag another person into this, I was at the Mensa World Gathering years ago and attending a talk by Deborah Ruf about interpreting IQ tests. One of the audience members asked her about improving one's intelligence or IQ through practice or training. Her response (I'm not quoting, just paraphrasing what I recall) was that you can increase your IQ test score by practicing the tasks on the test, but not your actual IQ or intelligence.

These seem to be totally divergent beliefs on intelligence, no?

Cricket! Thanks! Very good information. Your quotes/research does tease out the achievement/intelligence paradigm of these instruments, and those who utilize them.

I think, in general, these instruments are very sensitive to environment and ability of person administrating. I wish that were better teased out!

In NYC they use the OLSAT, or did, for gifted programs because it is easy and cheap to administer and fair for those who cannot pay for IQ tests. I was astounded that it showed less gifted in the group in the Canadian study since so many kids score above 97th percentile in NYC.

I disagree with Ruf's view. They have shown that IQ points increase with string instrument study. They got 6-7 IQ points from one to two years of string instrument study and it was sustained and increased math scores.

Now you can be born with advantage, like being a poorer athlete than Joe next door, but you take Parissi training, improve your technique and it stays with you. You become a better athlete than Joe next door. You can say you were not born with the natural DNA or maybe you just weren't using it properly. Does it make a difference if the scores improve and sustain?

I believe, and hope that she'll forgive me if I'm misinterpreting, that Dr Ruf's thought was that you could improve the IQ score, perhaps even in a sustained manner with continued practice, but you would not be changing your actual functioning in day to day life in terms of your intellect and ability to do things that require that intellect.

One thing that I would like to see more of in these studies is a greater breakdown of the correlation between measures for various groups. I.e. - the overall correlation was x, but what was it for subgroups and, in those that so have that, how many false negatives were there vs false positives.

Wow what a treasure trove of research you've shared with us this morning!! I'm excited to wake up to all the work you have done here.

Thanks so much for opening this discussion. So many gifted programs strictly use the CogAt and refuse outside testing that we really do need a developed body of research to aid when scores vary so widely.

I get that schools want cheap testing and need to eliminate push parents who want little Suzy in whether she needs it or not... but yes, what about false positives?? And what of the Canadian study where HALF of the qualified applicants were not identified?

I also wonder if there is a predictable group or set of factors which correlates with being misidentified on CogAt. If THAT research were available it could allow schools to continue with their cheap group testing while offering the option for other testing to be used in that cohort only.

Makes we want to start a grad program in psych testing and start doing that research!

I don't really believe in "false positives."

I do think that what you get is a snapshot at the time. I think time of day , child's mood, child's feelings about the tester, the tester's competence all figure into how a child proceeds. I don't understand how a child could falsely score high. It would be sheer luck and guessing and fooling the tester (I am talking about individually administered tests). I do think that you can "improve" your score with knowledge of the test. I really don't think how you respond to a new structure of assessment should be a part of the test score.

My child took the SCAT with no preparation, and didn't get the score cut-off the 1st time. The second time (re-test), we went over the structure of the exam, and I answered a few questions, and he did practice questions. The next SCAT, his scores were good enough for the cut offs at 2 grade levels above the already above grade level test.