0 members (),
179
guests, and
45
robots. |
Key:
Admin,
Global Mod,
Mod
|
|
S |
M |
T |
W |
T |
F |
S |
|
|
|
|
|
|
1
|
2
|
3
|
4
|
5
|
6
|
7
|
8
|
9
|
10
|
11
|
12
|
13
|
14
|
15
|
16
|
17
|
18
|
19
|
20
|
21
|
22
|
23
|
24
|
25
|
26
|
27
|
28
|
29
|
30
|
31
|
|
|
|
|
|
|
|
Joined: Apr 2011
Posts: 1,694
Member
|
Member
Joined: Apr 2011
Posts: 1,694 |
I have been pondering this thread all afternoon. I am not a psychologist or a statistician, so I might be thinking about this ALL wrong. But if the norming sample of the WPPSI-4 was 1700 children and they are using a normal curve it's just not possible for more than 1.7 of those children to score 145+, for more than 17 (total) to score 138+, for more than 34 (total) of those children to score more than 130. Does anyone know how many gifted children were directly recruited for this test? There are a good number of parents that have cropped up on this board feeling cranky, and I see posts elsewhere on the internet, and from what I can make out these kids were recruited on the premise that they had either already tested as gifted (highly even?) or showed clear signs of being HG+ OR had a PG sibling. So they took 20, 30, 40,...100 (I have no idea exactly how many) potentially HG kids, plus any wild card kids in the "normal" kids they recruited and we're surprised that the "Flynn effect is more dramatic in the gifted population"? Am I missing something here? As I said, so not a psychologist or statistician....
I don't doubt that scores of kids tested early in the life cycle can be seen as "underestimates" versus kids tested midway and the last kids as being prone to "over estimates" but is deliberately recruiting groups of gifted to PG kids messing the initial data up somewhat?
Last edited by MumOfThree; 04/13/13 02:54 AM.
|
|
|
|
Joined: May 2009
Posts: 2,172
Member
|
Member
Joined: May 2009
Posts: 2,172 |
I'm on my phone vs a computer so I'll be brief, but I don't think that having gifted+ kids statistically over represented necessarily means that they will all be squeezed into a normal distribution thus falsely depressing the scores of all but the few highest scorers. I'd need to look into it further, and will try to later on a computer if someone else doesn't beat me to it, but I'm not sure that the goal with the norming group is to have a bell curve where there is only 2% of the group scoring in the top 2% of the test's ability level otherwise they really should have just taken a random sample of kids not intentionally recruited outliers.
|
|
|
|
Joined: Jul 2012
Posts: 1,478
Member
|
Member
Joined: Jul 2012
Posts: 1,478 |
Here's the norming ref on the WPPSI-IV: -- Updated normative sample standardized on 1,700 children ages 2:6–7:7 -- The normative sample was stratified to match current U.S. census data based on sex, race/ethnicity, parent education level, and geographical region for each group
Once they've normed and made their curve, that's it. Anything else becomes procedural modifications, like creating extended norms to address ceilings.
During test creation, questions can be tuned against specialized samples. Like you run the test for 60 kids who are evenly distributed in the 130 to 160 range (based on another test) and you correlate specific questions to those IQ ranges to create questions tied to higher end skill nuances. Testing outside of the normative sample is not included in the final curve.
|
|
|
|
Joined: Feb 2011
Posts: 5,181
Member
|
Member
Joined: Feb 2011
Posts: 5,181 |
Here's what I've always wondered-- just in GENERAL terms-- about IQ testing, norming, and yes, the Flynn effect.
aren't most of the children evaluated using such tools probably above the actual population mean?
The bottom line is that norming assumes that the sample evaluated during the validation process is representative of the entire population. I'm pretty sure that is NOT the case.
Most parents of kids who would/should score in the 90-115 range probably see little REASON to submit their kids for testing in the first place, barring other learning disabilities or the suspicion of them.
So yes, I question the validity of modern norming, which depends upon recruitment and modern practices of informed consent (unlike very much older tools which didn't require such stringent parental consent... or, for that matter, 'informing').
Schrödinger's cat walks into a bar. And doesn't.
|
|
|
|
Joined: Apr 2013
Posts: 14
Junior Member
|
Junior Member
Joined: Apr 2013
Posts: 14 |
I have always appreciated Dr Ruf's work in the field but I just have a feeling that her blog posts on this topic are more a marketing exercise than a ground breaking discovery. She has posted two blog posts in the past month on the topic and made numerous responses to people asking questions and the one constant is that her Ruf Estimates Kids IQ Test solves the problem she has discovered.
The issue she has discovered may well be a real problem but I think it is inappropriate, with little data to back up her claims, to subtly encourage parents to spend more money on testing (particularly on her test) to ensure they get an accurate result.
More appropriate would be a discussion about what other reasons there may be for the changes in scores, or acknowledgement that she has only been testing on the new tests for a couple of months, so a clear picture is near impossible.
I am sure all her followers would benefit more from a balanced discussion and less sales pitch.
|
|
|
|
Joined: Apr 2011
Posts: 1,694
Member
|
Member
Joined: Apr 2011
Posts: 1,694 |
I noted that too and am leary. But it's the first time I have seen a professional say publicly what I have tried to explain that I am sure is the case to friends more than once...
|
|
|
|
Joined: Apr 2013
Posts: 14
Junior Member
|
Junior Member
Joined: Apr 2013
Posts: 14 |
Flynn Effect aside, an obvious concern is comparing the SBV with the WPPSI-IV. Scores vary between tests for a wide variety of reasons, it would be more relevant to compare the WPPSI-III with the WPPSI-IV.
Just thinking of my son, he scored extremely high in quantitative reasoning on the SBV but I know his processing speed is notoriously slow. If they replaced quantitative reasoning with processing speed as one of the subtests I have no doubt his scored would have dropped at least 10 points, throw in the Flynn Effect and we are at least 15 points lower. If he was 15 points lower than what the SBV score says he is now he would struggle to qualify for a gifted program. Instead, because he took the SBV, he qualifies easily while doing year 1 math after he comes home from pre-school.
All that aside, she may well be on to something. Time will tell us more.
|
|
|
|
Joined: Sep 2008
Posts: 1,898
Member
|
Member
Joined: Sep 2008
Posts: 1,898 |
Given this comment from Ruf below that blog post I'd be reluctant to believe anything she says about the Flynn effect! The same is true, by the way, for kids who need support services at the low end of the ability spectrum, e.g., the scores drift away from average at that end, too. This means that [when the identification test is new] a child with learning difficulties, a slower learner, will score HIGHER than the cut-off score needed for him to qualify for special education help in school. Flynn himself uses the example of death row cases where prisoners whose IQ's are too low are considered incapable of understanding that what they did was wrong; they're spared the death penalty. But when tests are new, more of them miss that low-end cut-off and are put to death. (She has it backwards.)
Email: my username, followed by 2, at google's mail
|
|
|
|
Joined: May 2009
Posts: 2,172
Member
|
Member
Joined: May 2009
Posts: 2,172 |
Like the last few posters, I have some mixed feelings on Dr. Ruf's writing. I really do agree with a lot of what she writes in this blog post, although ColinsMum is right as far as I've always understood that using newer tests nets lower scores for everyone including lower scorers, but I am also really big on something beyond subjective data when labeling kids. I follow Dr. Ruf on Facebook and, if you look @ her recent posts there, she also writes: The more I learn about how unreliable IQ test scores can be, the more I realize that descriptions of young gifted people are still probably the most reliable way to figure out how to meet their needs. Yes, I do believe that IQ scores can be off. I have one child who has never been tested except by a doctoral student, so who knows if those scores were too low, too high, or just right. I have another who is wildly 2e and whose IQ scores range from MG to PG, but who is always at least visibly gifted to some extent on IQ tests with there always being parts at the ceiling. I guess that my point on this is that, while IQ tests aren't the be all end all, I do believe that there should be something in the IQ test that indicates giftedness to believe that the child is gifted. I don't think that throwing the baby out with the bath water, so to speak, is the way to go just b/c there are issues with IQ tests including tester error, Flynn effect, etc. I also strongly believe that, when we are dealing with a label that, for better or worse, is viewed as "desirable," people are going to develop a bit of hypochondria when looking at "symptom" lists or behavioral signs of giftedness. Oh yes, that mole does look slightly uneven, I must have cancer! Oh yes, my child is sensitive, she must be gifted! The latter of those two statements I see being very likely among parents who want their children to be something that society views as a positive. I'm not trying to link people to my writing so I'm just going to post the whole thing here in a quote below if you want to read it, but I wrote an article on this last summer. Perhaps I've just become inured to parents overdiagnosing their kids with giftedness b/c I've been around too many parents for whom getting that GT id is a parental pissing contest, if you'll pardon the phrase, but I don't believe that anything but IQ is a better means of diagnosing giftedness at this point even if IQ is imperfect. Schools throughout the United States, including those in Colorado, frequently utilize subjective measures as part of their criteria for identifying children as gifted. How accurate are measures such as behavioral rating scales in ascertaining whether a child is gifted? The research does not strongly support rating scales as an accurate means of identifying gifted children.
While it may be true that most gifted children exhibit characteristics such as sensitivity and perfectionism, the reverse is not as true. All children who are sensitive and perfectionistic are not necessarily gifted. It is much like the mathematical analogy that all squares are rectangles, but all rectangles are not squares.
Locally, in [local] Districts, a child can be identified as gifted with a combination of any two of the following items: a behavioral rating scale such as the SIGS, high grades or other performance measures, high achievement on tests such as the [NCLB tests], or an ability test such as a group ability test or an IQ test. IQ tests usually are not administered and are only considered when private testing has been done.
So, what does the research show? The well known and local Gifted Development Center's (GDC) work on behavioral characteristics of giftedness is often cited as proof that parents are good at identifying whether their children are gifted. What the GDC studies found was that, using the Characteristics of Giftedness Scale that they developed for use in their own work,
"parents of all the children who scored in the gifted range indicated that their children manifested at least 13 of the 16 characteristics in the original scale"
and
"In this study, 84% of the children whose parents indicated that they fit three-fourths of the characteristics tested above 120 IQ."
Neither of these two statements is the same as saying that parents or teachers who identify students with behavioral characteristics such as curiosity and a good memory, two of the characteristics on their scale, have identified which children are gifted.
The first statement indicates, again, that all squares (gifted children) are rectangles (possessing certain characteristics), not that all children who possess those characteristics are gifted. The second statement indicates that 84% of parents who believe that their children are gifted strongly enough to spend over $1000 having an assessment of their aptitude done at the GDC were close enough to right that their children had IQs at or above the 91st percentile.
In terms of the behavioral scale most commonly used locally, the SIGS (Scales for Identifying Gifted Students), it shows only weak to moderate correlation with intelligence scores (.38 to .67 as correlated with the Weschler Intelligence Scale for Children III / WISC III) and it has not been correlated with the most recent version of the WISC, which itself is so outdated that it is about to be renormed. A correlation coefficient of less than .5 is considered a weak correlation and the correlation coefficient should be greater than .8 to show a strong relationship between variables.
What needs to be kept in mind when using subjective measures such as behavioral rating scales is that they should not be given as much weight as more quantitative measures of giftedness such as ability testing. Additionally, when dealing with what is a often considered a desirable label such as "gifted," teachers with no formal education in giftedness and even parents can easily misidentify non-equilateral rectangles as squares.
|
|
|
|
Joined: Feb 2011
Posts: 1,432
Member
|
Member
Joined: Feb 2011
Posts: 1,432 |
I can't say that I am surprised. Even looking at academics 5 to 10 years apart in the same school and even with the same teacher, I can see the significant upward push in the reading/langugage arts area. However, the changes seem even more profound with the preschool set - from the Einstein baby videos to the leapfrog educational toys and other electronics.
|
|
|
|
|