Gifted Issues Discussion homepage Forums Identification, Testing & Assessment A tale of two tests: SAT&ACT

I don't know that there is any real evidence to back this up-- or if anyone has actually looked at this effect at all, even-- but I have to wonder if HG+ kids tend to do increasingly worse on some types of achievement tests as they outgrow them. Well, "outgrow" isn't the term that I'm looking for, here, but basically what I mean is that if the SAT is "appropriate" as a tool for measuring a particular 6th-7th grader, then that same student as a high school sophomore is probably taking the SAT as a "below level" multiple choice assessment.

Okay, with that in mind, I feel that the ACT is much better-constructed. It's.... just...

well, the questions that I saw in prep materials were tighter. Less ambiguous or open to interpretation in different levels, if that makes any sense.

That's why I think that my DD probably found the ACT a better reflection of where she is NOW.

At 12, the PSAT was already somewhat below-level for her-- and it showed as she took practice tests. Her responses to looking up wrong answers wasn't; "Ohhhh, so THAT's how you do that..."

It was more along the lines of "Ohhhh-- so THAT is what they wanted me to read into that..."

Which, in my mind, is a mark of a test that is too level-locked to be very good for students below or above that level of cognitive function.

She was simply not reading the questions the way that a sophomore with the assumed background would do; aside from the grammar and math sections (which are very right-wrong and non-ambiguous), the test items were a struggle for her since she had to put herself into the position of "what would my classmates be expected to read into this item?"


For an example of what I'm getting at here, try taking a quick reading practice yourself-- with both ACT and SAT. I do much better on the ACT version, because it doesn't assume that I have any particular scaffolding beneath me to support interpretation, and therefore it's less prone (IMO) to over-thinking. KWIM? In looking at her practice tests for any patterns (particular types of questions, etc.)-- there wasn't a pattern of what she missed, usually-- I often found myself scratching MY head at the SAT items, and pretty much never with the ACT ones.

Could just be my family, too. I'm a zen-master of the standardized test, and DH and DD not so much. I tend to test probably better than I should, and both of them about as they should or a bit worse than that on the day.

I don't know about the ACT, but my impression of the SAT is in line with what HK's dd reports. Looking at SAT prep books, the SAT math is fairly easy theoretically and computationally, but the problems often involve a series of calculations and a twist or trick at the end. So, for example, perhaps the hard part of the question is that you have to factor a quadratic equation and it takes awhile to do so, but the question asked in the end is not the value of x or y but is (due to a twist in the question) actually 1/2 x. Maybe that's not a good example, but what I'm getting at is that I and my dc's tend to see a math question and jump to what's hard and what's (usually) important, which in this case would be doing the factoring right. We might process it so quickly that we don't read the extra sentence at the end saying that the value requested is 1/2x. That's why the SAT gives an advantage to preparation: you will have seen that trick a million times and will watch for it.

The tests have also differed in scoring: The ACT has not traditionally had a penalty for guessing, whereas SAT has.

And by guessing, they mean "guess and be wrong" as opposed to "guess and be lucky"?

DS does not like tricky. He'll make things tricky enough himself, and he "intuits" too much especially in math. This happens in class, too. (His chemistry teacher calls him The Bond Whisperer for intuiting covalent bonds.) Or, he'll read something different into a question than was intended.

Thank goodness the SAT essay didn't matter this go-round, because the question was so odd, it was leading and unclear at the same time...not to mention, they just haven't done that particular kind of writing in school. Still, the scorers gave him an okay mark...and yes, I read the "confessions of an SAT essay scorer". We can view the scanned copy, which is of such poor quality I can't read it.

I confess, I did not take the practice tests with him. He took them, I scored the bubbles, and he went over the answer guides (where he found the publisher errors). Very hands off for me (except for contacting the publisher and the library where we got our book).

I suppose we will wait and see whether he takes the SAT again, next time for reporting purposes. Of course, by that time the new test will be in place. Most students around here take only the ACT.

It's been quite a while since I've taken either one, but this is what I remember feeling after I took both tests in high school. On the SAT there was a lot of "I think this is the answer that they want, but that answer is making assumptions which may or may not be true." I felt that I did much better on the ACT because I didn't spend so much time wondering if the test designer was making logical leaps.

You have lots of good advice and suggestions above. I'll just throw one additional variable into the mix - any one test is just that, one test. Outside factors that have nothing to do with the specific knowledge required to perform well on the tests might come into play on any given day. When I think of SATs the first thing that comes to mind is a vision of the room I was sitting in when I took it.... something like eons ago. Same for the GRE. I don't remember anything about the ACT test setting. The reason I remember the SAT and GRE tests is that the environment was totally not conducive to being able to concentrate - on my GRE, there was a phone ringing in the next room - non-stop - that couldn't be taken off the hook because the door was locked and none of the test proctors had the key. So I wouldn't discount the possibility that something in the environment on test day impacted the score.

polarbear

According to one study SAT scores are higher at test centers with fewer students. A summary "New Research: Taking the SAT in a Crowded Room Means Lower Scores" is at http://www.nurtureshock.com/NEffect.pdf , and here is the paper abstract:

http://pss.sagepub.com/content/20/7/871.abstract
The N-Effect
More Competitors, Less Competition
Stephen M. Garcia and Avishalom Tor
Psychological Science July 2009 vol. 20 no. 7 871-877
Abstract
This article introduces the N-effect—the discovery that increasing the number of competitors (N) can decrease competitive motivation. Studies 1a and 1b found evidence that average test scores (e.g., SAT scores) fall as the average number of test takers at test-taking venues increases. Study 2 found that individuals trying to finish an easy quiz among the top 20% in terms of speed finished significantly faster if they believed they were competing in a pool of 10 rather than 100 other people. Study 3 showed that the N-effect is strong among individuals high in social-comparison orientation and weak among those low in social-comparison orientation. Study 4 directly linked the N-effect to social comparison, ruling out ratio bias as an explanation of our results and finding that social comparison becomes less important as N increases. Finally, Study 5 found that the N-effect is mediated by social comparison. Limitations, future directions, and implications are discussed.

Here is an interesting article regarding test performance (which raises many questions for a follow up research study): APA - 2007 - Seeing red impairs test performance. I'd be curious to know how they controlled for other variables among participants (breakfast, sleep, etc).

LOL, yes! Penalty for wrong answers. At least until biometrics are assessing whether a correct answer was made with certainty or by eliminating some possibilities and then guessing. (Chilling)

Interesting. Here are the possible mitigating factors the day DS took SAT:

-- could not find EITHER of his acceptable calculators. Solution: borrowed the identical calculator from our neighbor.
-- could not find EITHER his school or state IDs. Solution: he used his travel id. BTW, we handle always these things calmly, matter-of-factly, no frantic search or "WHY don't you have them" type drama.
-- test was in a nearby school he'd never been to. But, he did see a friend/classmate.
-- classmate was in his test room. Hmm, is there a competition factor? DS is not competitive, but the friend is.
-- the test was at least an hour and a half longer than the ACT due to the mandatory essay, but the essay came last.
They put about 20 kids to a room, so it's nice and small but you are totally aware of how many finish before you.

On the other hand, the first time he took the ACT in 6th grade, he was self-conscious about being the youngest kid there though he did see a friend, he was one of the few not writing the essay, and was taking it the same time as his HS junior brother, and he did very well.