Gifted Issues Discussion homepage Forums GT Research Are gifted education programs a waste of money?

http://worthwhile.typepad.com/worth...fted-education-programs-need-reform.html
In my latest Globe and Mail piece, I summarized a study by Sa Bui, Steven G. Craig, and Scott Imberman on the effectiveness of gifted education. The authors look at students in a large urban American school district who were evaluated for gifted programming in grade five. They ask: Who does better on the grade 6 and 7 standardized tests, the students who just made it into the gifted program, or the ones who fell just below the gifted threshold?

The authors have an impressive amount of data: standardized grade 5 test scores for 5,500 students either side of the gifted cut-off point before the gifted programming begins, and the same students� grade 6 test scores, one year later. They have similar information for 2,600 grade 7 students.

To test the effectiveness of gifted education, they measure how far each student was away from the gifted cut-off. The authors then estimate grade 6 and grade 7 standardized test scores as a function of distance from the gifted-eligibility threshold and some other controls. A "regression discontinuity" analysis is used to figure out if those who make it into the gifted program experience a jump in educational outcomes.

It's easier to explain with a picture than with words: [graph at site]

For reading and language - the green line and the red line - there is no jump in the test results at the gifted threshold. There's a kink, but not a shift. From this, the authors conclude that �students exposed to gifted-talent curriculum for the entirety of 6th grade plus half of 7th grade exhibit no significant improvement in achievement.� This is despite the fact that the students in the gifted-talented program have more educational resources coming their way � they are in classes with higher performing peers, are more likely to be placed in advanced classes, and more likely to attend a gifted-talented magnet school.

The lack of improvement in reading and language can be explained in a number of ways. The less able "gifted" students might feel discouraged by being in the bottom of the class and thus put less effort into school. The standardized test scores shown in the figure above might be measuring innate ability rather than what is taught in school. Reading and language scores may be more influenced by home environment than what is taught in the classroom.

But what is really striking is the suggestion that math results actually *fall* for those identified as gifted.

<rest of article at link>

The study is also discussed at http://www.nationalreview.com/phi-beta-cons/267179/end-gifted-programs-robert-verbruggen and http://giftedexchange.blogspot.com/2011/05/gifted-education-not-smart-idea.html .

I certainly don't think well-run gifted education programs are a waste of money, but advocates of gifted education will need to respond to the paper of Bui, Craig, and Imberman.

I read this blog last night about what good teaching looks like. �I'm not sure how correct the ideas are since I don't know anything about public school education but the blog was very entertaining and convincing.
http://instructivist.blogspot.com/

This described one of my kids. He took a grade-level standardized test last year without any review (he was accelerated in math by two years relative to grade level). His math scores were average to slightly above. This year we did some review (not a lot) and his scores were in the 9th stanine. Sample size of one, but it makes sense.

Emm. On closer inspection of their graph, I'm questioning the conclusion that math scores "fell" among GT kids. The "fall" doesn't look like much of a fall to me. It actually looks like it's about 0.05 of a standard deviation. In easily-understood IQ terms, this would be a difference of less than 0.75 of an IQ point. This is a difference? "I am smarter than you. You have an IQ of 124.25, whereas mine is 125."

I didn't see the original study and have to get my kids right now, but I'd be interested in knowing how they discussed this apparently very small difference and how much emphasis they put on it significance WRT special programs for gifted kids.


The scores increase steadily with increasing distance above the threshold (and decrease likewise in the other direction). So the study shows pretty clearly that smarter kids got higher scores on tests of the same academic material.

Given this observation, it's not too surprising to me that people whose IQs are nearly identical might have test scores that are nearly identical.

(Of course, I'm speaking in aggregate terms here: on average, the smarter kids did better. But some smarter kids who didn't study probably got lower scores than a less-smart kids who did study.)


Philosophical note: the authors were "test[ing] the effectiveness of gifted education," but they were making an assumption that standardized computer-read multiple choice tests are a good measure of the success of education.

I disagree. These tests don't measure a student's ability to synthesize knowledge or think of new solutions. They're focused exclusively on answering relatively straightforward questions quickly. They may be a crude measure of processing speed. But I don't think they measure "education."


This statement "might" be an exaggeration in the sense that saying "The ants were bigger than a 1954 Chevy and they ate my parents" might be an exaggeration.

Them! Them!

Viewing it in terms of money and test scores very much reminds me of the conversations of parents I hear IRL as well as online regarding NYC and the testing which everyone complains about. Here there is so much focus on what the gts get[i][/i] meaning the specials, chess, mandarin, Singapore math. Basically the argument is that if all kids had that stuff they would be doing better and love school too. There are two camps about that - the my kid deserves that stuff too camp - and the they all catch up in 3rd grade so it doesn't matter anyway camp. Some will focus on the they move faster argument, other on the depth argument. Here there is almost a rejection of the need while saying my kid has to have it. The lake woebegone effect as it were.

My personal view has changed so much watching the development of my kid. A year and half ago, I wanted a nice private because of the small class size, then I started to realize more was needed, so then the local gifted became my focus, then the citywides as each time my DS forced me to realize these options will likely not entirely serve him. But still it's better than being gen Ed, where he would always be waiting. What is the price for not waiting - if you look at private plane costs, apparently it's worth a very large sum!

DeHe

Andrew Sullivan ran a link to the same article and got a lot of flack from people over it. At least he listened.

I fundamentally disagree with the premise of the study. My kids are not in gt classrooms to improve their test scores. They are there to get work at their level, have a peer group and get extra social and emotional support. Any gt program focused on test scores sorely misses the point.

Maybe one problem is with the idea of GT programs. My understanding is that many of these programs are brief pullouts one day a week rather than meaningful approaches to meeting the needs of gifted kids. Everyone here knows that gifted kids generally need to move faster and/or go deeper.

Schools could fix the "move faster" part of this problem at very low cost by just scheduling the same classes at the same time in lower grades (this solution would also address the needs of kids who do okay in reading but need to repeat last year's math). A private school around here found a way to get kids from the middle school to the high school for advanced classes. It is simple. They put them on a shuttle bus. Other private schools (and our local charter school) manage to teach geometry and other HS subjects in 8th grade. Most or nearly all public schools outside of very rural areas could do this if they wanted to. But I think that many simply do not.

Oh well.

Val, my kids are in full-time gt classrooms, not pull-outs. I guess that is another question -- what kind of gt programming model were they using in the study? Were they full-time classrooms or pullouts?

Wow, your kids are lucky. Do the classrooms accelerate the kids?

They are accelerated in math and language arts. They still do grade-level science and social studies curriculum but supposedly at a "deeper level." I know that their teacher will let them test out of grade-level curriculum and then do something related that is more interesting/engaging.

Interesting. The paper is available for free.

Here's how the kids get into the GT program:


Admission to the district GT program is based on a set of points gained through scores as follows:

• A standardized test (Stanford Achievement, aka SAT) (56 possible points)
• The non-verbal portion of the NNAT (30 possible points)
• Your report card (20 possible points)
• Teacher recommendation (10 possible points)
• Whether or not you have obstacles (15 possible points)
• Total is 131 points

You qualify for the program if you get...62 points. See page 35 of the manuscript for the form showing the points.

This could be achieved with the following:

• SAT scores from 70th-79th percentile in all subtests (12 total points)
• The second lowest score band (104-107) listed on the NNAT (10 points). Not sure what 104-107 means in terms of percentiles. Anyone know? I read in one place that they correspond roughly to IQ distribution scores. Is this correct?
• 90-94% on the report card (15 points)
• Top teacher recommendation (10 points)
• Three obstacles (15 points)
• Total is 62 points

So, this pretty much means that average kids can qualify if they get good grades, their teachers like them, and get extra points for the obstacles listed on the form.

The study compared kids close to the cutoff on both sides. So it compared average kids in a gifted program with other average kids not in a gifted program.


Gotta go.

Maybe it's not quite that simple, Val. You've chosen a particular combination of scores that you feel is most non-gifted, but some other kids without obstacles etc. would achieve the cutoff score with, say, a higher SAT score. Not only that, but the very next page of the report shows that kids at the cutoff point have an 80% chance of receiving G/T services. Which of the 62-pointers do you think are more likely to receive services-- the well-liked kids with obstacles who are completely underwhelming academically, or other ones? ETA: I see now that it's based on a lottery. Of course.

I generally agree with a lot of the rationalizations in this thread, though. From what I can tell, a lot of G/T services are along the lines of enrichment that wouldn't necessarily result in increased achievement scores. I don't think that's the way it would be in an ideal world, but that's probably the way it is a good deal of the time.

The scores I picked were noted in the paper; I should have mentioned this but I was late for collecting kids:


So in reality, scores can be even lower than 62 for the GT program. In fact, the scores seem deliberately set up to catch kids with the 70th percentile scores on the SAT, 104-107 on the NNAT, 80-84% on their report cards, a strong teacher recommendation, and 15 bonus points. Add it up on page 32: you get 56.

By way of comparison, an IQ at the 70th percentile (15 point SD) is 108. A 35 year-old white woman who is ~5'5.5" tall is at the 70th percentile. For a white man, it's 5'11". Taller than most but not super-tall. Stuff clusters very tightly around the average when you have a normal distribution of data, and even 20 percentage points above average is still not too far away from there. (You would expect a normal distribution for scores on standardized tests; they make them that to be way).

For comparison, one standard deviation (84th percentile/IQ of 115 or height of about 5'7" or 6 feet) IS starting to move away from the average. Two standard deviations is an IQ of 130 or a male height of 6'3." Now THAT'S tall by white people standards.

IMHO, a score of 56 or 62 on this scale are very, very low for the purposes of defining someone as "gifted." This is merely kind-of-maybe-above-average.

Note also that you get three bonus points just for being in a minority group, so the qualifying score gets pushed down a bit more for some of the students.

Sure, you could play with the numbers and use high scores in one area and low scores in another. But you could also push some scores below the 70th percentile (0 points) and put some in the next score band to compensate up to 62.

The point is that the authors claim to be reporting on "high achievers," but they weren't. They were focusing on average-ish achievers.

The school district clearly set up the cutoffs to catch a lot of average-ish kids, and they're the ones most likely to score at the low end of this particular scale. Sure, there were probably a few gifties in the group the paper studied, but most of them probably weren't. Why? Well, if you define gifties as only 3% of the population, then at most a small number of the entire GT group would be gifted. Then remove anyone with a score over 72, as the authors seem to do, and you lose a lot of the gifties.

So the real question they studied was: "Did inclusion in a GT program in a large urban school district raise test scores among average students?" Their answer was "No."

Of course if the study is really measuring only the impact of GT programming on average kids, one might be tempted to say, "Who cares?" I'm happy if that's not cost-effective, if limited GT resources should be spent on GT kids. My question was really not whether the GT program and study caught a lot of average-ish kids, but whether the study measured enough above-average-ish kids at the cutoff to make the results meaningful in the intended way. Unfortunately, we don't know anything about the giftedness of the kids at the cutoff; we just know their matrix scores. We certainly can't guess that the scores on that silly matrix map to IQ, and certainly not to intelligence, whatever that is. (Says the guy who skimmed exactly two pages of that report so far.) The study may well be too flawed to be meaningful at all, I guess, although it's not so clear to me just from the logic you've presented here.

It additionally occurred to me that if we were to accept that GT programming should result in higher achievement scores, then a failure to improve achievement scores of kids at the boundary would still be cause for a valid complaint about the overall cost-effectiveness of the program-- it wouldn't matter how gifted those kids were, and it wouldn't be necessary to disprove an effect on more highly gifted kids. That is, the bar being set too low for identification and entry could be a big problem, and such a study might help point that out. And if the GT programming would theoretically result in increased achievement scores for higher-achieving or smarter kids, I see no reason for just assuming that the effect would peter out completely before reaching kids at the cutoff; that would be cause for investigating further instead of making a questionable assumption.

I don't have time to read the whole study. I'm left wondering why, since they must have had data available for the higher achievers too, they would focus just on the boundary kids (ETA: It looks like they must not have had enough data based on attrition rates of high achievers, and the fact that most high achievers stayed in the GT program). To that extent you've heightened my skepticism of the study. Still, a program doesn't have to be a complete waste to be wasteful. Note also the key words "well run" from the OP.

I will have to think further about whether or not gifted programming should result in a measurable effect of some sort. It seems desirable to have some sort of metrics about the success of a program, in addition to reports of happy students, although ordinary achievement test scores might not be the proper metric. I agree with Bostonian that gifted kids deserve to be taught, in addition to deserving to be socially and emotionally happy. I also don't like throwing money down a hole. (Of course, I live in an area with no gifted programs, so the whole discussion, as my most-hated boss would have said, is "mute" for me.)

The authors of the paper do consider this issue (p19 of the pdf http://www.class.uh.edu/faculty/simberman/bci_2011.pdf):

"One particular concern is that the lack of results may be due to top-coding of the exams.
Since GT students are high-achievers many of them may not be able to exhibit growth on
achievement tests as they are very close to getting every question correct. To address this, in Online Appendix Figures 7 � 11 we provide distribution plots of raw scores on each of the 7th grade Stanford Achievement Tests for students with Euclidean distances between -10 and 10. In all cases the mass of the distribution is centered quite far from the maximum score. For example in math the modal score is 62 out of 80 while for reading it is 67 out of 84 leaving substantial room for improvement."

But the kids with "Euclidean distances between -10 and 10" are precisely the average students with low test scores that I mentioned. They aren't high achievers and it's no surprise that there's "substantial room for improvement" on a grade level test.

Hmm. I read up on this recently and wrote something about it for work that synthesized a large pile of research on the subject. There is plenty of evidence out there that accelerating, skipping, and early entrance to K or college are beneficial for GT kids. Enrichment programs have the least evidence in their favor, but IIRC, there is at least some. All this media furor is focusing on this one study, largely because the conclusion is "exciting" and controversial and makes for a good chance to throw rotten fruit at the "pushy parent." Meanwhile it ignores decades of much better research. Nothing is about this is atypical about science reporting. But of course, to regard this study as "proof" that there is no point to ANY kind of gifted education, or even as proof that there is no such thing as giftedness, is throwing the baby out with the bathwater in a major way.

Pull-out programs vary enormously, in my experience, and it doesn't surprise me at all to learn that some have no effect on test scores. My pull-out program in elementary was pretty great, but by high school it was useless "enrichment" that did absolutely nothing for my skills, though it was mildly interesting.

Our gifted program seems to be excellent. It starts in 4th grade. The kids in 4-6th grade cover 4 years of math in 3 years, so they eventually take Geometry in 8th grade, Algebra II in 9th grade, Pre-Cal in 10th grade, Calculus AP-AB in 11th grade, and Calculus AP-BC in 12th grade (if you want to go that route).
So the gifted program kids end up skipping 1-2 years of math over the regular program. I'm sure the English part is similarly accelerated.
I don't think that is a waste of time or money.

Why aren't they using above grade level test to measure growth?

I don't think so either. The progress of the kids in that program also would show up on achievement tests, coincidentally (again, I'm not saying that that is necessarily the only way of measuring success).

I believe that you find what your looking for, If the author of this study wanted to find it was a waste of money, he would pull information that supported his idea. My experience leads me to believe that gifted education can be effective if the school supports it with a teacher that wants to teach these children and an adminstrator that values and believes that these children need appropriate education. I have not seen great results with pullout programs. There is just not enough time to address the children's needs in a half hour three times a week and too much repetition in the general ed. classes. The only way to provide effective education is a self contained gifted class targeting the needs of the children in the program.

The graph is pretty interesting.

Looking at the plots for math, I see that the non-GT program kids cluster along the line, but for the kids in GT-program, quite a few have developed a .1 to .2 SD separation from the line and the other kids, which suggests that for them - the GT program has had quite an effect.

Another explanation for the continuity between the two groups and which explains the high-achievers is that some kids are already being pushed to their limit and this sets the curve in both - and these kids will not accelerate their learning because they are already saturated. For kids who are not at the edge of their ability to learn, they will "bump up" after being in a tougher program - hence the outliers.

It should be easy to set up a "gate" to cut out the kids who are saturated to make the one who benefit stand out. You could then do a regression test on some other aspects to then come up with a way to find those who will benefit the most.

That program is typical of the well run school districts in the DFW area for "honors" track students. IMHO a truly accelerated program would see Calculus done in the 10th grade, with Diff-EQ+Multiavariate Calculus/Real Analysis in the 11th and Linear Algebra/Abstract Algebra in the 12th. I'd add in a Logic class in the 10th as a prep class.

For physics, we'd get the normal HS stuff out of the way by the 10th, then jump into statics/kinematics in the 11th and a full electromag in the 12th.

You could teach stats in a good physcial chem class in the 11th then do organic chem in the 12th.

Round it out with lit and history classes that have a heavy literary criticism, public speaking, and essay part.

Add a third year with summers, and the students could have a BS out of the way then be ready to get certs in a given field, finish up upper division specific classes at the U, ie Engineering or Accounting degree, or go into grad school.

I tend to agree with you. A gifted math student has wasted a lot of time if they only find themselves in Calc I or Calc II by high school graduation. I think a good, compacted math curriculum would teach pre-algebra and algebra in 1 year. The next year would cover geometry and trigonometry. Followed by algebra II and pre-calculus in one year. This puts the student in calculus in 10th grade assuming they covered algebra in 7th grade. If they do a B/C curriculum, they're ready for Calc 3, linear algebra, or diff-eq as a Junior. A properly motivated, gifted math student may even want to take more than 1 math course a semester, and why not accommodate them? As I recall, Calc III, differential equations, linear algebra, and probability/statistics are not prerequisites for each other.

I find the science sequence at the high school dd12 is set to enter a bit frustrating for the same reason. You can't take AP bio until you've taken pre-AP bio; you can't take AP chem until you've taken pre-AP chem, etc. Basically, you have to take 2-3 periods of science every year in order to get into the higher level science classes.

Dd is signed up for pre-AP bio and earth systems science her freshman year but worries, in looking @ the books, that they'll be quantity over learning anything new. She had hoped to test out of the earth systems science course, which they used to allow, but they no longer allow that. It leaves very, very little time for other electives to take so many full year science classes in order to reach the higher level courses.

If she is just interested in learning the material and taking the exam for college credit, she can study AP Bio and AP Environmental Science for free any time here:

The Hippocampus

It might free up some room in her schedule for electives.