Gifted Issues Discussion homepage Forums GT Research The Elite Illusion (public exam schools)

http://papers.nber.org/papers/W17264
The Elite Illusion: Achievement Effects at Boston and New York
Exam Schools
by Atila Abdulkadiroglu, Joshua D. Angrist, Parag A. Pathak - #17264 (ED LS)

Abstract:
Talented students compete fiercely for seats at Boston and New York
exam schools. These schools are characterized by high levels of peer
achievement and a demanding curriculum tailored to each district's
highest achievers. While exam school students clearly do very well
in school, the question of whether an exam school education adds
value relative to a regular public education remains open. We
estimate the causal effect of exam school attendance using a
regression-discontinuity design, reporting both parametric and
non-parametric estimates. We also develop a procedure that addresses
the potential for confounding in regression-discontinuity designs
with multiple, closely-spaced admissions cutoffs. The outcomes
studied here include scores on state standardized achievement tests,
PSAT and SAT participation and scores, and AP scores. Our estimates
show little effect of exam school offers on most students'
achievement in most grades. We use two-stage least squares to
convert reduced form estimates of the effects of exam school offers
into estimates of peer and tracking effects, arguing that these
appear to be unimportant in this context. On the other hand, a
Boston exam school education seems to have a modest effect on high
school English scores for minority applicants. A small group of 9th
grade applicants also appears to do better on SAT Reasoning. These
localized gains notwithstanding, the intense competition for exam
school seats does not appear to be justified by improved learning for
a broad set of students.

******************************************************

The paper is at http://econ-www.mit.edu/files/6856 . This will be a paper that advocates of gifted education may not welcome but will need to respond to, as was the paper discussed in the thread "Are gifted education programs a waste of money?" http://giftedissues.davidsongifted....2430/Are_gifted_education_programs_.html

Economists are so cute, with the assumptions they make, and the way they try to apply their fanciest math to the behavior of adolescents! Yes, that's what we need to analyze how well a school is doing--more sophisticated math!

[quote]
Our strategy in a nutshell is to compare the scores of exam school applicants
who barely clear the admissions cuto to the scores of those who fall just below.
[quote]
It may well be that being a big fish in a less competitive pond is actually better for outcomes like SAT score then being the littlest fish in the most competitive pond. I think it depends on the student. It may well be that the measurable effects would be on the students who score the highest going into a selective school - because they have been sandbagged the most. Of course finding good outcome measurement for that group is tricky - probably not enough headroom in the SATs to demonstrate some very real differences.

((shrugs))
Grinity

As someone who attended a fiercely competitive exam school, and is also 2E, I would really hate to have my school experience reduced to my ability to perform on the SAT! My SATs were good, and good enough to get me where I wanted to go to college, but it had nothing to do with my school. My SAT scores went up due to Princeton Review - my achievement tests scores are entirely related to actually attending my school and the fabulous educational opportunities finally experienced in high school.

I understand the rationale of comparing those who got in with those who did not in terms of data that is readily available to the researchers. But it totally misses the point of these schools - what these schools provide is the opportunity for breadth and depth of subject in an environment where it is acceptable to be smart and to enjoying learning. the SAT tests none of that.

Letting go of enormous pet peeve now and stepping down of the soapbox . ..

DeHe

The paper is discussed at the Boston Globe:

http://boston.com/community/blogs/rock_the_schoolhouse/2011/08/historically_many_of_massachus.html
Do exam schools add value?
by Jim Stergios
August 7, 2011 10:31 AM

Obviously SAT scores were a bad place to look for differences in education between schools.

What's interesting is that analysis failed to reveal a difference in the number of AP tests taken, and passed, despite the fact that the exam schools offer so many more AP courses. It seems that instead of taking more tests, exam school students are simply selecting a similar number from a larger domain. I don't see that the study attempted to make any distinction between taking and passing a Calc B/C test rather than a Calc A/B test though, which seems like a flaw to me.

On the other hand, is it reasonable to expect that it's no better to be in a class where the pace is a little too fast than a little too slow?

If we want to know the value of exam schools, shouldn't we focus on the students who are more typical of exam schools? The students who got in (not borderline) but didn't go because of geography, etc?

I whole-heartedly agree. They also provide the experience of going to school with a group of peers of similar ability. It probably gives many of them the experience of being "average" for the first time in their lives -- something I didn't have until I went to college. I think that the social and emotional benefits of such an environment should not be discounted.

Another recent paper finding similar results:

http://www.people.fas.harvard.edu/~dobbie/research/ExamSchools_July2011.pdf

Exam High Schools and Academic Achievement: Evidence from New York City
Will Dobbie, Roland G. Fryer, Jr.
NBER Working Paper No. 17286
Issued in August 2011
Abstract: Publicly funded exam schools educate many of the world's most talented students. These schools typically contain higher achieving peers, more rigorous instruction, and additional resources compared to regular public schools. This paper uses a sharp discontinuity in the admissions process at three prominent exam schools in New York City to provide the first causal estimate of the impact of attending an exam school in the United States on longer term academic outcomes. Attending an exam school increases the rigor of high school courses taken and the probability that a student graduates with an advanced high school degree. Surprisingly, however, attending an exam school has little impact on Scholastic Aptitude Test scores, college enrollment, or college graduation -- casting doubt on their ultimate long term impact.

Except.

They used Bronx HS of Science v Stuyvesant as the means to check for a discontinuity.

The programs at Bronx are every bit as good as Stuyvesant and all three schools used are exam schools - so I do not see this as a good test.

If you wanted to test, you would have to send the kids who did not get into the exam school to a "normal" school.

I would think one way to do this would be to track kids who moved out of NYC to schools that produced just a handful of high SAT scoring students each year.

An interesting history of New York's exam schools is

http://www.city-journal.org/html/9_2_how_gothams_elite.html
How Gotham�s Elite High Schools Escaped the Leveller�s Ax
Heather MacDonald
City Journal
Spring 1999