Gifted Issues Discussion homepage Forums Identification, Testing & Assessment Scoring WIAT-III into IAS-3 (Achievement)

Hi all. I have a technical question, which I hope someone here has the experience/expertise to answer.

I recently approached my daughter's school about possible grade-skip acceleration (K -> 2). The school policy is to use the Iowa Acceleration Scale (IAS-3) to assess candidates. The school psychologist administered some psychometric tests, choosing the WISC-V (for Ability) and WIAT-III (for Achievement). All good, until daughter's Achievement score came back way lower than expected (score 3/8 in Section VI of Iowa Scale). ALL previous grade-normed tests consistently put her in 99th percentile in every subject.

Because I was curious about why the Iowa Scale score for Achievement was not the expected 8/8, I looked into the detail of the psychologist's report, and it appears to me (with a background in statistics and social sciences) that the WIAT-III results were translated into the IAS-3 in a very odd way. I have read the IAS-3 Manual. The psychologist is defensive and not willing to discuss it.

So my question is, to anyone with expertise (or access to any relevant official policy/procedure), how should they have scored Achievement in section VI of the IAS-3 from WIAT-III data? What scores should be used from the available WIAT-III results, which includes the sub-test scores, 5 composite scores and the total achievement score. WISC-V results are also available (and were used to score Section IV. Ability of the IAS form).

I'll fill in later the details on how it was actually done in our case, as I don't want to pre-empt the answers...

thanks in advance,
William

Not knowing what your specific quibble is, I'll describe both what you might expect from standard uses of the IAS, and some possible irregularities.

Academic achievement testing typically is scored both using age norms (the actual age of the child) and grade norms (the grade of the receiving grade, or one or two grades up--in this case, using first grade norms), to see how they compare normatively both with their age peers (these are true norms) and with their prospective peers, since the objective is to place them into the upper end of the receiving grade, not the struggling end. Both results would be reported as standard scores or percentiles (or some other transformation of z scores). So there should be two places where the same WIAT-III results might be entered, but using different numbers.

Achievement: based on her current grade (or age, if she is typical age-for-grade right now). These would be expected to look much more like what you've experienced in the past. Typically schools are looking for results at 90+ %ile (or higher, depending on the local population; 95th %ile is also common).

Aptitude: based on the grade of the receiving cohort. These would be expected to be substantially lower than you've historically seen. Typically schools are looking for 50+ %ile (75th %ile is also fairly common, again depending on local population).

There are some who interpret performance at least two grade levels up as meaning grade equivalents from achievement testing two grades up. While this is not an unusual interpretation, it is, as you may be aware as a person with some statistical sophistication, not a psychometrically sound one. The problems with using grade equivalents with instruments like the WIAT-III have already been thoroughly discussed numerous times, including elsewhere on this forum.

It may be that your school system has instituted policies for acceleration that require personnel (i.e., the school psych) to work through the IAS criteria in a way that may not be psychometric best practice, which may or may not be something the psych agrees with, but certainly isn't something they could comfortably disagree with publicly, let alone to the parent of a student in the decision-making process.

Whatever the nature of your doubts about application of test data, until you know where the decision to apply them that way originated, it will be unclear whether, how, or with whom, to address them.

Thanks aeh for taking the time to respond, and thanks for the clarifications. My problems seem to have arisen fromt here being a policy vacuum here - the school system says that the Iowa Acceleration scale is to be used, but provides no further guidance - leaving it for school psychs to figure it out for themselves.

Sorry, I was rather vague in my original question. My specific issue is not with use of the WIAT-III or the (grade) norm used, but rather with how the WIAT-III results were translated into the Achievement section (Section VI) of the IAS-3. This is what was done:


IAS-3 Section VI (Achievement - max 8 points)

'Vocabulary' (2 points) - scored solely from the 'Vocabulary' sub-test score of the WISC-V

'Total Reading' (2 points) - scored solely from the 'Reading (Early Reading Skills)' composite of the WIAT-III

'Total Math' (2 points) - scored solely from the 'Mathematics' composite of the WIAT-III (in isolation, this one actually makes sense)

'Total Language' (2 points) - scored solely from the 'Oral Language' composite of the WIAT-III

'Other' (2 points) - Not used, despite there being unused composites from the WIAT-III available.


The 'Total Achievement Score' from the WIAT-III was available and reported by the psych, but ignored for IAS purposes. Ditto the 'Written Expression' composite. As a non-expert, I would imagine that if the WIAT-III is the instrument chosen to measure achievement for IAS purposes, then the appropriately weighted, full-scale 'Total Achievement' score that the WIAT-III provides is the most valid and reliable measure? What rational could there be for not using the full-scale score in for this purpose?

Likewise, written expression is presumably an important component of total achievement, so I'd contend it's not a good idea to exclude it? (important in our case, as daughter's standard score for this composite was 140, 99.6th %ile - one of her highest composites, but not taken into account at all!)

The use of WISC-V in the Achievement section seems contrary to the explicit instructions in the IAS manual (i.e. it is an Ability test rather than an Achievement test). And then there are the statistical issues with recycling the same data in two sections, and of inflating just one WISC-V sub-test (out of 10 taken) to score fully one quarter of achievement...

So my specific question is, what is the validity of this approach to scoring achievement in the IAS? It appears that the psych has not had the opportunity to read the IAS-3 Manual and related literature closely. The approach taken appears to have been to look at the categories suggested on the IAS-3 form, and against each scored whichever composite/subtest score most closely matched the name of the category. There is no evidence of consideration being given to whether or not this approach might actually reflect the child's academic attainment. My reading of the IAS-3 Manual suggests the categories are guides only, to help in cases where data from different sources needs to be combined.

Unfortunately the school psych is unwilling to specifically address any of these points, or revisit their method.

Am I mistaken in my belief that the approach taken is of questionable validity, and contrary to the intention of the IAS?

W

My reading of the manual is that the scores applied to reading, math, and language are within conventional usages. (There is not a better measure of reading on the WIAT-III, due to her grade.) I personally probably would have considered the receptive/expressive vocabulary component scores (probably the higher of the two) from the WIAT-III for the vocabulary measure, since, as you say, the WISC measure has already been included in the ability section, but I can understand the school psych using the WISC-V measure, as it is probably a deeper measure of oral vocabulary than the two achievement measures are (the WIAT-III components are both versions of picture vocabulary, which tends to restrict vocabulary to a more concrete range). For all we know, perhaps that was the highest indicator of vocabulary of the three.

One could make a legitimate argument that the intent of Total Language is either oral language or written language. If the former, then the OL composite is appropriate. If the latter, then the WE composite is appropriate. Again, given her age, choosing OL has a rationale behind it, as the WE composite is purely mechanical at this age (alphabet writing and spelling), and both does not plumb as deep into actual language skills, and also may disadvantage academically-advanced young children with still-developing handwriting skills. (Using OL here probably also explains why measures from this composite were not used for the Vocabulary section; they'd be double-counted.) This probably depends mainly on district policy, though. "Other" is in the same category. If district policy has not defined how Other is used, then the SP is within rights not to apply any score to it, so that candidates are compared to a common yardstick.

Total Achievement combines measures from all aspects of achievement, which has the result either of obscuring focal strengths, or simply duplicating information that has already been entered in the specific academic domains, much like your concern about reusing the WISC-V Vocabulary subtest.

BTW, I assume they administered additional subtests (six beyond the ones you've named) to complete the Aptitude section? At age six, there are age norms for all of the subtests but essay composition and math fluency, and, in any case, they would be using the grade one or two norms for the aptitude measures. If they did, there would be Oral Language, Total Reading, Basic Reading, Written Expression, and Mathematics Composites available for the Aptitude section. You report five composites in addition to Total Achievement.

The school psych's defensiveness, honestly, is irrelevant. (Full disclosure: I'm in the same profession, and have been in similar positions wrt GT decision-making.) Perhaps I would make different professional decisions in his/her case (I can't say, without knowing the specifics), both with regard to scoring and communication, but nothing you've described appears glaringly inappropriate.

Hi aeh.

Thanks for those insights.

I am left with two fundamental questions:

1. Is there any guidance from the IAS author, or any other authoritative source, about how the category names in Section VI are to be interpreted or translated from other instruments? (e.g. are they to be rigidly applied, or just suggestions?

2. Is there any evidence or theory-based rationale for using a sub-set of achievement scores in this context, instead of a full-scale measure if available, like the WIAT-III Total Achievement score?

regards

W

1. Remember that the IAS is first intended to be a jumping-off point for thoughtful, collaborative discussion and decision-making by the team of family and school, not necessarily to be a robust psychometric measure. In any given large institution, it is probably more important that the category names are used consistently system-wide, than that their interpretation matches an idealized IAS.

2. Given that the sections describe distinct academic skills, the Total Achievement Composite, which blurs those distinctions, is unlikely to be the optimal measure.