The Pleasures Of 'Teaching To the Test' - Gifted Issues Discussion Forum

http://online.wsj.com/news/articles/SB10001424052702304104504579374651890320212
The Pleasures Of 'Teaching To the Test'
By JAMES SAMUELSON
Wall Street Journal
February 12, 2014

Is standardized testing anti-student? Many educators and commentators believe so, vehemently. No more "drill and kill," some detractors demand. Kids are not robots goes another refrain. Others argue that standardized testing is a soul-sapping exercise in rote learning that devalues critical thinking and favors students of higher-income parents who can afford test-prep classes or private tutors.

On the contrary: Testing is good for the intellectual health of students. It is also an excellent way for teachers to better understand the particular academic challenges their students face.

First, standardized tests are a critical thinker's dream. Multiple-choice questions often ask students to evaluate evidence and make inferences. Consider a sample multiple-choice question for the New York State English Language Arts test, which is administered in the public schools. It asks students to identify the tone of a paragraph excerpted from Andrew Carnegie's "The Gospel of Wealth" (1889).

Students must closely read the author's choice of words and phrases so as not to choose plausible but incorrect suggested answers such as "humble," and instead zero in on the correct response, "confident." The ability to do so takes intense focus, stamina and, perhaps most importantly, practice.

Questions such as these are not based on a test-taker's ability to memorize facts�a major criticism invoked by test-taking opponents�but a student's analytical prowess. Close reading to determine the connotation of words and phrases is not merely a test-taking skill. It is a skill needed for a fulfilling, literate life. And it's a skill that students can learn, if teachers are willing to teach them.

**************************************************

I think much of the criticism of standardized tests on this forum, especially of the multiple choice variety, is overdone.

Originally Posted by Bostonian

First, standardized tests are a critical thinker's dream. Multiple-choice questions often ask students to evaluate evidence and make inferences. Consider a sample multiple-choice question for the New York State English Language Arts test, which is administered in the public schools. It asks students to identify the tone of a paragraph excerpted from Andrew Carnegie's "The Gospel of Wealth" (1889).

I think much of the criticism of standardized tests on this forum, especially of the multiple choice variety, is overdone.

I don't see the validity of this point. MC tests ask students to pick from four predetermined choices. They don't require him to figure out the answer for himself by putting ideas together (my definition of critical thinking). Rarely in life are we presented with four answer choices, one of which is correct.

Asking a student to pick from a few choices about the tone of a paragraph is a far cry from asking him to write a meaningful essay about it.

Indeed, Val. I think the author needs to become a better critical thinker before mistaking such a question as a "critical thinker's dream." For one thing, the author describes this as "evaluating evidence," but where's the part where the reader is encouraged to say, "this passage presents bad evidence, because _________."

And this only scratches the surface of the real issues with standardized testing, which relate to how well they measure what they purport to measure, and how the results are misused to evaluate teachers (because, news flash, all students are not alike).

If all of this multiple choice testing is so "good" for intellectual development, then why has the SAT needed recentering with ever-increasing frequency?

It isn't because scores have RISEN over time.

`Recentered' Scores Just Another Step Toward Mediocrity

Average SAT scores by year since 1952

Average Scores Slip on SAT - NYTimes, 2011

National Center for Ed Statistics-- SAT score data

So when SHOULD we start to see this strategic shift toward testing and "critical thinking" (the new definition above, I mean) paying off?

I mean, if it WORKED... then shouldn't the scores of students who were high school juniors and seniors in, say, 2011-- having been the beneficiaries of a full K through 12 education aligned toward this (IMO dubious) mission be demonstrating a huge JUMP in scores?? Yet that seems to emphatically NOT be the case. If anything, the only real result seems to be in writing, where scores have declined. Big surprise.

Like a lot of other untested "great ideas" in education, this one is actually kind of an epic... failure... when one examines the DATA to see how well it actually works.

Sort of like math instruction via computer. The longer students spend without real instruction, the worse they do. Not a popular result, btw, and therefore one that those pushing technology and automation into classrooms have studiously IGNORED.
Executive Summary:Effectiveness of Reading and Mathematics Software Products

Originally Posted by HowlerKarma

If all of this multiple choice testing is so "good" for intellectual development, then why has the SAT needed recentering with ever-increasing frequency?

It isn't because scores have RISEN over time.

`Recentered' Scores Just Another Step Toward Mediocrity

Average SAT scores by year since 1952

www.nytimes.com/2011/09/15/education/15sat.html

So when SHOULD we start to see this strategic shift toward testing and "critical tihnking" (in this new definition I mean) paying off?

I think the changes in SAT scores are mostly driven by the larger number of kids taking them and going to college. The SAT scores of the top 60% of HS kids is bound to be very different than the SAT scores of the top 20-30% of HS kids that used to take the test.

Quote

I don't see the validity of this point. MC tests ask students to pick from four predetermined choices. They don't require him to figure out the answer for himself by putting ideas together (my definition of critical thinking). Rarely in life are we presented with four answer choices, one of which is correct.

Asking a student to pick from a few choices about the tone of a paragraph is a far cry from asking him to write a meaningful essay about it.

Hear, hear!

We've already taught DD10 the extremely useful but (IMO) not especially "analytical" skill of "If you aren't sure, cross out the obviously stupid answers and pick from the remaining ones." And the "backwards plug-in" (in math, plug the answers you have left after crossing out the dumb ones into the problem and see if they look like they work). These are test-taking hacks. One could argue they are applicable to other things in life, but they aren't demostrating mastery of the material.

Would someone like John Nash emerge from standardized testing or advocate for its use? I vehemently think not!

If we require kids to perform well on a test, then we better teach them how to do so.

It infuriates me when kids are expected to perform well and educators state "we don't teach to the test".
I agree that teaching should be more than teaching to the test, however, teaching to do well on the test needs to be included. Otherwise, we are teaching about oranges, testing about apples, and then complaining that we got oranges.

Maybe a better idea is to teach authentically and then assess based on what is being taught in classrooms.

In other words, I think that fundamentally, we're already asking all the wrong questions to begin with the instant that we start "teaching TO the test."

We should TEST to the TEACHING.

Yes to curriculum-based assessment and teachers should be using it to assess the quality of their teaching.

Schools in most (all?) other OECD countries teach to the test. But the tests are very different.

For example, in Ireland, everyone know that the test would have questions from Jane Eyre and A Tale of Two Cities, so the schools taught Jane Eyre and A Tale of Two Cities. This was done by reading the books, writing papers about them, and talking about them. It was not done by reading excerpts, constructing paragraphs (11 sentences each!), and choosing the best answer about Mr. Rochester's reaction to his house burning down. Etc. Their tests have few multiple choice questions (if any), but there are essay questions about the books and long-answer problems in math/science. Questions generally take 10-20 minutes to answer.

But in this country, students have less than a minute per question. Everything is superficial. Read a passage and pick the best answer. What is the slope of the line? So the teaching becomes superficial, even though the deeper approach would almost certainly raise test scores.

The superficiality of our education system is destroying us. It's like the people in charge don't know about the general principles of different subjects and how concepts fit together.

Do we have anyone here that has a good understanding of the educational systems that do well either in Europe (Finland / Netherlands), or Asia (Korea, Taiwan or Japan)?

My very limited understanding is that the European model is quite different from the Asian model. Currently the Asian model is doing better on the PISA rankings, for what that is worth.

I've read extensively on Finland's school system. Their model is simple: Teachers are highly skilled professionals and are fully supported in that position.

It all ripples from there.

I know about Ireland (undergrad and grad student there for 5 years or so, friends with sibs in school) and Switzerland (grad student there; sister-in-law is a teacher there). Are you specifically interested in Finland and the Netherlands?

ETA to what ZenScanner wrote: the same is true in Ireland and Switzerland. My SIL is very smart and very dedicated and very good at her job. She's strongly supported by her bosses.

Ireland has a national curriculum and all schools get equal financial support. Switzerland's cantons set their own curricula, but everyone takes the same test at the end of high school. All schools get equal financial support.

ETA to ETA: Very few kids go to private school in Switzerland. Well, very few kids whose parents also live in Switzerland.

Finland's PISA scores have been falling:

http://www.economist.com/news/inter...star-latest-pisa-tests-focusing-interest
Finn-ished
The fall of a former Nordic education star in the latest PISA tests is focusing interest on the tougher Asian model instead
The Economist
Dec 7th 2013 | From the print edition

Quote

WHEN the first Programme for International Student Assessment (PISA) tests to focus on maths results were published a decade ago, Finland�s blue-cross flag fluttered near the top of the rankings. Its pupils excelled at numeracy, and topped the table in science and reading. Education reformers found the prospect of non-selective, high-achieving and low-stress education bewitching.

Every three years since then, 15-year-olds have sat PISA tests in maths, reading and science. In 2012 fully 500,000 heads were bent over desks for the exam in 65 countries or cities. The results, published on December 3rd, doled out a large helping of humble pie to Europe�s former champion. Finland has fallen by 22 points on its 2009 result, with smaller falls (12 points and 9) in reading and science. �The golden days are over,� lamented the Finnbay news website.

None of this should have come as a surprise. Finland�s maths performance has been tailing off since 2006. But it is worsening faster than in other countries with falling scores such as Canada and Denmark. The Asian high-fliers (Shanghai, Hong Kong and Singapore) have consolidated their position at the top. Much soul-searching is under way in Helsinki.

Leena Krokfors, an academic there, blames declining motivation and a failure of maths teachers and the curriculum to inspire enthusiasm. Others are beginning to wonder whether the egalitarian nature of Finnish education might be an underlying problem. Juha Yla-Jaaski, who runs a technology project to stretch the academically able, worries that a focus on raising the achievement of the majority of pupils shortchanges the cleverest. The country is �kidding itself�, Mr Yla-Jaaski says, if it thinks they can catch up at university.

As Finns argue about how to retain their pre-eminence, many other countries in the West still envy it�as well as the progress of rapid improvers such as Estonia and Poland. France and Germany, in contrast, have flatlined. America�s dire showing led Arne Duncan, the education secretary, to decry �a picture of educational stagnation�, with Americans being �out-educated� by the Chinese. Some hope for a motivating shock like that delivered in 1957 by the Soviet Sputnik launch.

There's a couple of schools of thought with standardized testing between the US and other countries and compulsory public education:

1. Some people believe we're losing our global competitiveness to other countries, such as S. Korea and other Asian countries in particular. S. Korea, Singapore, and other Asian countries traditionally score high on standardized math exams.

2. Others, such as Diane Ravitch who initially supported CC and Bush jr, have reversed course and believe it's a bunch of poppycock. They say that the US in particular has never scored high on those standardized tests. Moreover, Ravitch says that standardized test score argument is weak when we consider the overall creativity and originality of US patents, technologies, or companies such as Google and Facebook.

Here's Ravitch's blog on it - http://dianeravitch.net/2013/12/03/my-view-of-the-pisa-scores/

3. Some say Finland and northern European (sans UK who seem to be caught up in national testing and standards) take a more hands-off approach to reading and other standardized tests and this ultimately breeds more successful students. Denmark, on the other hand, has been widely touted as the happiest people on earth for a number of years.

4. Some in China, such as Zhao Bowen, are now trying to locate and isolate high intelligence to spawn superbabies. Read this article - http://dianeravitch.net/2013/12/03/my-view-of-the-pisa-scores/

In order to breed a Leonard da Vinci, though, you need to have the mind of a scientist AND the mind of the artist. In the US, we tend to forget that second part and that's to the country's detriment, imo. You need that divergent thinking that comes with music, the arts, creating, the humanities, etc.

IMO, we need to take a much more individualized, personal approach to education - more like unschooling. This isn't going to happen though. But the harsh reality is that we need friendly, kind car mechanics who are technical whizzes but don't test well on standardized tests as much as we do people who score off-the-chart on those standardized tests. And I was really cursing public education for ditching auto shop when I needed a car mechanic!

Sorry, China article on why people are so smart and creating superbabies - http://www.wired.com/wiredscience/2013/07/genetics-of-iq/

Note - Zhao Bowen is still looking for an IQ genetic marker yet others have argued that exceptional working memory is a better indicator for education and potential (and prodigies).

Originally Posted by cdfox

Others, such as Diane Ravitch who initially supported CC and Bush jr, have reversed course and believe it's a bunch of poppycock. They say that the US in particular has never scored high on those standardized tests. Moreover, Ravitch says that standardized test score argument is weak when we consider the overall creativity and originality of US patents, technologies, or companies such as Google and Facebook.

Oh dear.

I wondered how many of those patent holders came from elsewhere, and if she only counted American-educated inventors.

Someone else asked the same question in the comments section. The reply was what I would expect from someone who DIDN'T control appropriately:

Quote

Jim, December 8, 2013 at 8:12 pm

�And when it came to creativity, the U.S. �clobbered the world,� with more patents per million people than any other nation.�

Well, if we�re using this measure to rate this country�s education system, we should exclude patents gained by people who received their education outside the US and immigrated. I�ll bet that�s a lot of them.

dianeravitch December 8, 2013 at 11:30 pm

Jim, why did they immigrate here? Freedom? Opportunity? Great universities?

Can anyone Name That Formal Fallacy?

Seriously, this what edumacator types do when you catch them trying to distort something: change the subject.

Originally Posted by Val

Ireland has a national curriculum and all schools get equal financial support. Switzerland's cantons set their own curricula, but everyone takes the same test at the end of high school. All schools get equal financial support.

It's easier to reach consensus on a national curriculum, including which specific books are to be taught, in a relatively homogeneous country of about 5 million such as Ireland than in a sprawling, diverse country of 300+ million such as the U.S. Imagine if all the countries in the European Union had to come up with a continent-wide curriculum.

There are substantial differences in academic achievement across states. People bemoan the U.S. PISA results, but Massachusetts PISA results are among the world's best. If you made the students in Mississippi pass the same test (the MCAS) as students in Massachusetts do to get a high school diploma, the number of high school drop-outs produced would be unacceptably high. Common Core is an effort to fix the problem of differing state standards, but NCLB allowed states to set their own standards to avoid high failure rates in lower-performing states.

Switzerland has 4 official languages and they talk about adding English from time to time. It has long been a popular work destination for foreigners (like me). Currently, 27% of the population is foreign.

They come from Africa (French speaking cantons), the Middle East (all cantons), northern Europe (German-speaking cantons), southern Europe (Italian-speaking cantons), and a variety of other places. Geneva and Vaud are an international as places can get.

Quote

Moreover, Ravitch says that standardized test score argument is weak when we consider the overall creativity and originality of US patents, technologies, or companies such as Google and Facebook.

I think that many of those patents either belong to foreign companies, or have inventors in America that were educated overseas. For example, Sergey Brin was born in Russia.

Originally Posted by Val

Originally Posted by cdfox

Others, such as Diane Ravitch who initially supported CC and Bush jr, have reversed course and believe it's a bunch of poppycock. They say that the US in particular has never scored high on those standardized tests. Moreover, Ravitch says that standardized test score argument is weak when we consider the overall creativity and originality of US patents, technologies, or companies such as Google and Facebook.

Oh dear.

I wondered how many of those patent holders came from elsewhere, and if she only counted American-educated inventors.

Someone else asked the same question in the comments section. The reply was what I would expect from someone who DIDN'T control appropriately:

Quote

Jim, December 8, 2013 at 8:12 pm

�And when it came to creativity, the U.S. �clobbered the world,� with more patents per million people than any other nation.�

Well, if we�re using this measure to rate this country�s education system, we should exclude patents gained by people who received their education outside the US and immigrated. I�ll bet that�s a lot of them.

dianeravitch December 8, 2013 at 11:30 pm

Jim, why did they immigrate here? Freedom? Opportunity? Great universities?

Can anyone Name That Formal Fallacy?

Seriously, this what edumacator types do when you catch them trying to distort something: change the subject.

I agree-- but then we must also exclude foreign patents when THOSE individuals were educated overseas, as well.

In many STEM fields, relatively few Americans are educated abroad, and traditionally, a large number of the elite undergraduates from other nations are educated in the US.

When fully 2/3rd of a graduate program are here on educational visas, then it becomes quite difficult to tease apart just what is "theirs" and what is "ours."

Except, of course, for the fact that their students are somehow superior to so many of ours even though they are usually working ESL, at best.

And, um-- I could be quite wrong about this, but using Sergey Brin as an example of a foreign immigrant might not be the best idea, since he's got some pretty illustrious American institutions in his CV.

Now, what Larry Page and Sergy Brin both do have in common... is parents who are part of the intelligentsia... the Professoriate, at any rate. Clearly growing up in an environment that values education and provides appropriate opportunities (not limited by formal schooling and/or age) seems to be an important factor; this is also something that Bill Gates seems to have been raised with.

It's not at all clear that we'd have MORE people like them if only we offered more standardized testing, however.

I just want to say that I am loving this discussion.

good points...

Originally Posted by HowlerKarma

Now, what Larry Page and Sergy Brin both do have in common... is parents who are part of the intelligentsia... the Professoriate, at any rate. Clearly growing up in an environment that values education and provides appropriate opportunities (not limited by formal schooling and/or age) seems to be an important factor; this is also something that Bill Gates seems to have been raised with.

You wrote exactly what I had in mind, but I wrote it poorly above.

Originally Posted by HowlerKarma

It's not at all clear that we'd have MORE people like them if only we offered more standardized testing, however.

You get people like Sergey's parents coming to America as long as America remains a land of opportunity for immigrants. Standardized testing has no positive or negative effect on that.

Does standardized testing improve education for students in the US? Not by itself, but a well constructed test can measure how well someone has learned the material being tested. Measurement is highly useful, as it can provide insight about what is working and not working. In that sense, I think standardized tests are just as useful in great education systems as they are in ones that need improvement.

Last I looked, the copyright system doesn't solely benefit inventors or those who patent products. It benefits artists as well. In fact, Charles Dickens helped to push for the international copyright system to protect his work.

Ravitch's point, I think, is that the standardized test scores do squat for a country's creativity and ingenuity. There, I would agree with her. The American Revolution and Industrial Revolution started in MA and RI (not in NY, NJ, VA or elsewhere in the colonies) for a reason. Ideas.

Read her latest blog - http://dianeravitch.net/ which was written today. She asks whether Common Core will contribute to creativity or not. I think not. Many public schools have slashed art, music, gym, etc. A recipe for disaster, imo.

Bill Gates and MS and the big educational publishing/testing firms are touting standardized testing because it benefits them. It's good business. If schools keep Windows operating systems or MS, then schools continue being reliant/dependent on them and students will likely continue to use their products as future customers. Few question the status quo.

Of course, I'd also suggest looking at Scotland as an illustrative example of how a poor, sparsely populated country put value in educating its citizens. Scotland has been host to much inventiveness and for creating the basic ideas and institutions of modern life (Bell, Adam Smith, David Hume, James Watt, Carnegie, Arthur Conan Doyle, JK Rowling, etc.) - if we're keeping score!

Originally Posted by mithawk

Does standardized testing improve education for students in the US? Not by itself, but a well constructed test can measure how well someone has learned the material being tested. Measurement is highly useful, as it can provide insight about what is working and not working. In that sense, I think standardized tests are just as useful in great education systems as they are in ones that need improvement.

I agree. All the secondary school exit exams in other OECD countries are standardized: everyone takes the same test, and it's based on the same stuff. The difference is that other countries don't rely on multiple choice in the extreme way that the US does (some don't use them at all).

Re: patents.

My point was to show that her use of patents per citizen in the US was a (deeply) flawed way of highlighting how swell our education system is. I agree with points that have been made that aren't related to that conclusion, though.

So, here is my question. What would you say about those that test well on standardized test but have poor grades. Example a high above average score on a standardized test and a D in school?

It depends on what you know about the structure of both the test and the instruction which purportedly supports it.

It's possible that the test is far too easy, and that the instruction is genuinely rigorous and completely appropriate.

It's equally possible that the test is measuring mastery and that the student is completely checked out of the instruction because it is too remedial.

Also possible that it is neither of those things, and is a case of a curriculum and test that don't align in the least, and it's merely coincidental.

What is odd is that professors in Brin's generation (and my own) thought it was just fine to send their kids (no matter how able) to public schools.

That has changed rather radically in the last twenty years.

The shift seems to have been most extreme just in the past ten years.

HowlerKarma - yes, I agree with you on Brin's generation and the shift with the last 10 years.

Brin is the same age as my brother; I was in high school when Nation At Risk came out. Since then, we've had the same mantra about how public schools are not preparing students and falling behind in terms of global competition. Of course, since then, we've had one of the worst recessions since the Great Depression or in most people's memory. And, we've had major shifts for workers, the middle class, and an increasingly unpredictable economy.

In the 70s and 80s, you still could compare standardized test scores between school districts or states from the newspapers and people still relocated for better public schools. In MA, there was white flight to the suburbs and away from the inner cities or deprived areas, for instance. But you didn't have the Internet, of course. More significantly, you didn't have NCLB or other forms of state or national control to make teachers and public education more consistent or accountable, if you like.