I've thought a lot about standardized tests. I'm glad that you opened up this thread.
One aspect of multiple choice (MC) tests that drives me demented is that they can't measure sophisticated thinking. This is because students typically have an average of 60-90 seconds to answer a question. With this approach, there is simply no room for asking students to think about ideas, be creative/make connections, and solve something. They can't; there's not enough time. So they measure trivial stuff instead.
One thing I've observed is that the tests appear to compensate for the triviality by writing questions and answers in weird ways. This is especially the case on the verbal sections, and is more problematic now that analogies are gone from the SAT and GRE.
As an example, getting the right answer will hinge on noticing that a single word in the question or answer choice has a specific meaning, yet could be read in a general way. They accomplish this by using weasel words, synonyms, and vagueness. In other words, they try to trick you. This where one can get into trouble: if you don't recognize what's happening, it's easy to start analyzing and over-analyzing. In the MC test world of thinking, this approach is foundational (IMO) because there isn't time to ask you to write a meaningful essay or prove that something is equal to something else. If you can't grade it by running an answer sheet through a Scantron machine, it won't appear on an American standardized test. This is due to our love of industrial solutions to non-industrial problems.
Another point is that some of the questions in the verbal sections don't actually have bona fide correct answers.
This explains why some of passages can be clearly understood, yet the questions can be very difficult. It also explains why the directions say select the BEST answer, not select the RIGHT answer. As a concrete example, see the notorious question 7 in the
hare and pineapple passage.
Some years back, the testing companies came under fire for writing ambiguous questions. Rather than writing questions with answers that were clearly correct, their solution was to change the wording of the directions to "select the BEST answer."
You presumably know that the testing companies write questions and then test them on live tests (these are the unscored questions or sections). I don't know if this is true or not, but a commenter on the pineapple/hare debacle wrote that the "correct" answer for the ambiguous questions is the one that the most high-scoring students pick. Again, I don't know if this is true or not, but I've spent some time in the education area, and it certainly wouldn't surprise me if it were.
Compare with these exams:
Paper 1 and
paper 2 of the Honours level Irish Leaving Certificate in 2012. Here's
an English paper. These exams are graded by humans, not Scantrons, and the humans are typically academics (not low-wage people frantically reading SAT essays in two minutes or less with explicit instructions not to mark down for factual mistakes).
Read this. It will make you feel better.
