Gifted Issues Discussion homepage
Posted By: ColinsMum Freeman research vs A Nation Deceived etc. - 11/18/09 02:00 PM
Not exactly "recent" research, but stuff I've been looking at lately. Joan Freeman is well-known in the UK as an expert in giftedness. I've posted before that I have a bad feeling about her personally because of how she appears in the Channel 4 series Child Genius - however, I think my feeling is more of the "she reminds me of someone I didn't like" character than anything more rational, so I'm trying to read her work objectively.

Anyway, her main recent contention is that identifying children as gifted and treating them specially is a very bad thing. (Maybe you are starting to see why I then have misgivings about her participation in the TV series!) The research on which this is based was a longitudinal study starting in 1974 of UK children identified by their parents as gifted. The main paper I've been able to find about this is here:
http://www.joanfreeman.com/content/JEG%20Giftedness%20In%20the%20Long%20Term%2006.pdf
It appeared in the Journal for the Education of the Gifted in 2006. First question: what kind of journal is this? Is it a respected peer-reviewed journal? Its web page is here:
http://journals.prufrock.com/IJP/b/journal-for-the-education-of-the-gifted
but I couldn't immediately find a list of the members of its editorial board, which was a little surprising.
Here's an extract from the paper:

Quote
The Target group was 70 children aged between five and 14, described as gifted by their parents, almost entirely without testing, all of whom had joined the National Association for Gifted Children (the UK association is made up mostly of parents). Each Target child was matched with two Control children of the same sex, age and socio-economic level, sharing educational experience in the same school class. This careful matching enabled ability to be assessed on the Raven�s Matrices intelligence test raw scores, not the less-accurate percentiles. This group pattern test is non-verbal so that scores are very much less affected by home and school educational effects, and so is internationally widely used as a �culture free� test.
The First Control group was measured as of identical ability as the Target identified gifted children, though not labelled as such.

More questions, not addressed in the paper:
- isn't it nothing short of astonishing that it was possible to find such First Controls for the gifted children? (Identical Raven's score, same sex and SE class, and members of the same school class - a school class at that time would have been about 30 children.) Were there large numbers of gifted children who were excluded from the study because no control was available? If so, what about the way that biases the sample? If not, why not?

More info: once we have a target group of 70 children and a first and second control for each child, that's 210. She writes:
Quote
Of the whole sample, 170 children were at the 99th percentile of the Raven�s Matrices. Stanford-Binet IQs ranged from the 46 children with less than IQ120 to 18 children with above IQ160; 13 reached the Stanford-Binet test ceiling of 170 IQ.
Now the Second Controls were *not* supposed to be gifted - so the number of children who were at the 99th percentile of Raven's Matrices includes, apparently, all the gifted children, all the First Controls, *and* 30 of the Second Controls. (Or not all the first two groups and more of the last, of course). Huh? Maybe "the sample" is actually supposed to include all the classmates of all the 70 identified children? Even so, that would only be 2000-odd children, so 170 at the 99th percentile is still weird.

The rest of the paper I won't quote, but it amounts to that the labelled gifted children had more emotional problems and were no more successful than the First Controls, and it is clearly Freeman's belief that this is because of the labelling, although she refrains from stating this baldly. She does not seem to have researched the parents' reasons for identifying their children as gifted. Correlation is not causation! One has to wonder whether those parents were seeking help because they saw emotional problems in their children, so that the observed emotional problems were the reason for the labelling, not the other way round.

What surprises me - I'm an academic myself, though not in this field - is that issues like this would not be picked up by the referees of the paper, and explanations inserted before publication. I'd be very interested to know from those of you who are active in gifted research and/or have been reading it like me, how this work is regarded or what you think of it. Am I missing a more rigorous presentation of the same study somewhere? (I know she's written books, but those do not undergo peer review at all, so I'm less interested.)
Posted By: Dandy Re: Freeman research vs A Nation Deceived etc. - 11/18/09 04:27 PM
I came across this study earlier in the summer. My initial reaction was that this is an odd paper for someone who enjoys taking the labeling & special treatment of gifted children to the extreme.
Posted By: inky Re: Freeman research vs A Nation Deceived etc. - 11/18/09 04:51 PM
Thank you for posting the information and you bring up many good questions. I'm also astonished by the numbers for the study.

Did you see the Vita for the current editor?
www.bsu.edu/edpsych/media/pdf/tracy-cross-vita-02.05.07.pdf

Some of the points in the paper reminded me of Carol Dweck's research on growth vs. fixed mindset which I find more useful. http://www.dukegiftedletter.com/articles/vol8no3_feature.html



My thoughts on the study:
1) A label of gifted doesn't necessarily guarantee success later in life.

2) I think there may be a selection bias in the study, because the Target group was kids who were identified by their parents as gifted. Maybe a percentage of the kids were not gifted (say, as defined by objective measures), and the label of gifted was something these kids could never live up to. I'd like to see the study repeated with kids identified as gifted by tests, perhaps multiple tests, to assure the validity of the Target group.

3) Maybe there is a message to us as parents that labelling kids doesn't really help them. There are parallels to Dweck's work as Inky noted - not labelling the ability, but rather praising the effort.

Maybe we could add to the body of knowledge someday by enrolling our DC in a cohort study. How many kids are represented on this board? A few thousand? More?
From the first page of the pdf:

Quote
Better predictive factors were hard work, emotional support and a positive, open personal outlook.


There you go.

I can see her point about labelling. I think a better approach is to say "These kids learn much faster and in order to foster a good work ethic and a positive approach to life, we need to take this difference seriously and do X thus making them work hard and thus develop the emotional results from that."

I am reading the whole paper. Gross' results are mischaracterized.

Quote
In Australia, for 20 years Gross (2004) has followed up 10 boys and five girls originally aged 11 to 13, chosen because their Stanford-Binet IQs were more than 160. In general, she found the youngsters to have low self-esteem, �moderate to severe levels of depression�, not to mention �loneliness, social isolation and bitter unhappiness�
!!


Quote
The long parental interviews in their own homes disclosed that the labelled gifted children with emotional difficulties had significantly (p<1%) more problematic domestic circumstances, such as parental divorce or experiences which would disturb most children.

One cannot imply that the distinctly higher level of emotional and behavioural problems measured in these labelled gifted children were caused by their parent�s pressure on them. One can only report that this in-depth investigation discovered significantly more disturbing features in the home lives of the more problematic gifted children when compared with those of the non-disturbed equally gifted children. Using both the Stamford-Binet IQ and the Raven�s scores along with the rated data from the interviewing it was possible to see that it was not intelligence as such that caused these disturbances, but other matters in the children�s lives, (e.g. divorce, moving home frequently), and parental attitudes to their children�s upbringing (e.g. TV, homework, punishment, parental behaviour and beliefs). When asked why they had joined the NAGC (UK) for their children, most parents cited the children�s problems as typical of giftedness. The gifts often got the blame. Fortunately, as the children grew up and became more independent, most of these problems disappeared,

Quote
Sometimes far too much of the gifted young people�s energy had gone into fighting their school regimes and their teachers, supposedly there to help them. Too many had dissipated their time and energies into wrong channels because of poor educational guidance. At times, the youngsters told me that they knew exactly what they wanted to do, but were thwarted by reasons of school time-table or teacher opinion, and went into areas for which they were less well suited. One girl at a high-powered school, for example, was told that biology was not for her. Defying the school�s advice, she secretly entered a competition with her own biological research and won. Only then did the school recognise her potential and permit her to study in the subject area of her choice. She is now a research pharmacist.
Originally Posted by inky
Thank you for posting the information and you bring up many good questions. I'm also astonished by the numbers for the study.

Did you see the Vita for the current editor?
www.bsu.edu/edpsych/media/pdf/tracy-cross-vita-02.05.07.pdf

Some of the points in the paper reminded me of Carol Dweck's research on growth vs. fixed mindset which I find more useful. http://www.dukegiftedletter.com/articles/vol8no3_feature.html
I hadn't seen her CV, no. Looks good, but it takes more than one editor to make a journal! (In my field a journal normally has a couple of dozen members of the editorial board. A submitted paper is assigned by the editor in chief to a member of the board. The member picks referees for the paper, synthesises their opinions, and recommends acceptance/rejection or revision of the paper. It's a lot of work, even for a board member. No way can one editor do that job for all submissions.)

Carol Dweck has some very interesting papers on her web page:
https://www.stanford.edu/dept/psychology/cgi-bin/drupalm/cdweck
Thanks for pointing to her - I'd heard the name but not looked closely.
Originally Posted by twomoose
I think there may be a selection bias in the study, because the Target group was kids who were identified by their parents as gifted. Maybe a percentage of the kids were not gifted (say, as defined by objective measures), and the label of gifted was something these kids could never live up to. I'd like to see the study repeated with kids identified as gifted by tests, perhaps multiple tests, to assure the validity of the Target group.
That was my thought as well. The quote in the OP stated that these kids were ided as gifted primarily without any testing to support that contention. I do believe that there can be some harm to making assumptions that are not supported with data and kids who aren't gifted, but are being pressured to be high achievers b/c their parents have a need for them to be gifted for whatever reason, are likely to suffer some emotional fall out.
Originally Posted by inky
Did you see the Vita for the current editor?
www.bsu.edu/edpsych/media/pdf/tracy-cross-vita-02.05.07.pdf

I've met Tracy Cross. He is amazing in general, and 'gets' gifted kids. I wish I could say the same for all the people in the field.
Posted By: Chrys Re: Freeman research vs A Nation Deceived etc. - 10/15/10 01:15 AM
Joan Freeman has a new book out too - Gifted Lives. There has been some chatter on this board about it recently (because I asked if anyone else had read it.) I am still going to try to finish the book before I have an opinion about it, but right now it doesn't seem friendly.
I have been reading the boards for a while, but this is my first post! I too have wondered why Professor Freeman would on the one hand indicate that labeling a child as gifted might harm them, and on the other, taking part in the Child Genius programme where she gushed to both parents and the children themselves about how their IQ's were off the top of the chart. She can't have it both ways.
I was dismayed at the headlines in the UK press on the findings of Freeman's new research and the publication of her book. Here's one example:
Daily Mail UK so I wrote a blogpost about it too:
Irish Gifted Education Blog

Our gifted children have enough to be dealing with everyday without having to live up to the expectations of total strangers who are secretly hoping to see them trip up!
Originally Posted by BeamMeUp
Our gifted children have enough to be dealing with everyday without having to live up to the expectations of total strangers who are secretly hoping to see them trip up!
Well said, and welcome, BeamMeUp!
Thanks ColinsMum! Looking forward to learning loads more here, this forum is a phenomenal resource.
Originally Posted by BeamMeUp
I was dismayed at the headlines in the UK press on the findings of Freeman's new research and the publication of her book.

Dismayed is a good word for it, BeamMeUp! Lovely Blog post.
Reading the articles makes me realize that there is a lot about UK education I just don't know.
What are 'A levels?'
After high school, is there a public system of education for college?
Are there tests after high school to help kids document their abilities for colleges?
At the high school level, what are the main choices?

Thanks in advance,
Grinity
Originally Posted by Grinity
What are 'A levels?'
After high school, is there a public system of education for college?
Are there tests after high school to help kids document their abilities for colleges?
At the high school level, what are the main choices?

Thanks in advance,
Grinity

I'm not BeamMeUp, but can help with a couple of questions. It's funny, because I was thinking of asking questions about the US education system (mainly MS and HS, as I am clueless about how it all works over here). Anyhow - I was born and lived in England until 13 years ago, so can give you a quick idea of a couple of things. Most kids at about age 14 choose what subjects they are going to focus on for the next few years - I know when I was in school, I had to choose 8 subjects, I'm not sure how it works now (I left school in 1979). I did Math, English Language, English Literature, Physics, Chemistry, French, Geography and Religious Edication (!). From age 14 - 16, I focused on those subjects, and at age 16, took exams in them, which are now called GCSE's. (back then they were 'O Levels). Also, these days, the coursework they do during the year counts towards the final grade, I believe (in my day, it was all in the final exam). After 16, you can then go on and take 'A' Levels (another 2 years work) - I think the norm is 3 or 4 subjects, but some do more. You basically choose some of the subjects that you have already done in the previous 2 years, and expand on them - obviously, most people choose whatever will help them best in whatever they want to do when they go to University. I have heard that 'A' Levels are the hardest exams you will ever take - but I didn't do them, so can't verify that smile
Hope this is helpful - I'm sure BeamMeUp will get back to you and let you know more about the current state of education in England ! smile
Thanks NCPMom - wow, so you skipped 'history' for 2 years? Did you study each subject for both years? In High School many one year classes have a test called SAT II that is standard and can help a student show colleges what they have achieved. There are also AP tests, that correspond to 'college level' classes which are taught at half speed within the high school. Some colleges will give credit or at least advanced standing if the AP scores are high enough - but the most competitive colleges won't.

In the US there are public high schools that are open to everyone and supported by local taxes, as well as independent school (used to be called private) that charge tuition. Independent schools can be religious affiliated or not. Usually religious affiliated charge less tuition, sometimes about a tenth of the tuition of a non religious affiliated school, because they are supported by their organization. Independent high schools can be 'day' vs. 'boarding' and ones that focus on college bound kids can be called 'prep schools.'

Hope that helps,
Grinity
Good summary Grinity.

One thing I would add, in the US as in the UK, is that many independent schools have from 15-30% of their students getting partial or full tuition support from the school's endowment funded by the wealthier families, alumni, and local supporters.
This support enables families of limited means to enroll their children in schools previously reserved for the very well off.
This also raises the caliber of the student body overall and greatly enriches the learning for all the kids. A number of independent schools have seen their overall test scores rise as a result of the opening of the selection process to a larger applicant pool.

Most colleges in the US do have a test-out policy that allows for students to bypass prerequisite classes via testing. This is a different policy from granting credit for AP tests. Students will sit for the class final or a standardized test and must score over 90-95%.

There is also a test given in the next to last year of secondary education, or the 11th grade, called the Preliminary SAT. The top 4/10% of those test takers usually become National Merit winners and most state schools and many private colleges offer full or partial scholarships for these students.





Quote
Most kids at about age 14 choose what subjects they are going to focus on for the next few years - I know when I was in school, I had to choose 8 subjects, I'm not sure how it works now (I left school in 1979). I did Math, English Language, English Literature, Physics, Chemistry, French, Geography and Religious Edication (!). From age 14 - 16, I focused on those subjects, and at age 16, took exams in them, which are now called GCSE's. (back then they were 'O Levels). Also, these days, the coursework they do during the year counts towards the final grade, I believe (in my day, it was all in the final exam). After 16, you can then go on and take 'A' Levels (another 2 years work) - I think the norm is 3 or 4 subjects, but some do more. You basically choose some of the subjects that you have already done in the previous 2 years, and expand on them - obviously, most people choose whatever will help them best in whatever they want to do when they go to University.

Parts of Harry Potter suddenly make more sense to me. smile
Actually I'm across the water in Ireland, another system entirely...but we get a lot of UK press reported here, hence my interest in Joan Freeman's book and the articles which quoted her.

Our system is different again. At the start of secondary school, at age 12/13, students study about 11 subjects for their Junior Certificate Examination. English, Irish, Maths, a modern language, History, Geography, Science, Civic Social & Political Education (CSPE)and Religion are common obligatory subjects in many schools. Then you can choose another two or three from choices such as Latin, Art, Music, Classical Studies, Technical Graphics. This exam is taken at the end of Third Year.

Most schools offer what is known as Transition Year (TY) which is the 4th year of secondary school and in which students take a break from formal exam preparation. They have an opportunity to develop skills across a range of non-academic and academic areas such as volunteering, drama, work-experience etc. They continue to study core subjects of English, Irish and Maths, and some of their other subject choices.

In 5th and 6th Year students prepare for the Leaving Certificate Exam. Their results are calculated in grades which are each assigned a number of points. The points serve as entry requirements for university courses. Students usually take 7 or sometimes 8 subjects with the best 6 results counted for Points. For entry to Irish universities English, Irish and Maths is required, so almost all students take those. There are almost 30 subjects offered, although most schools can only schedule far fewer than that. Some students take an extra subject outside school, but this is considered a risk because the courses are content-heavy and the workload is considerable.

We have no formal provision for gifted learners here. It is prohibited by our 1998 Education Act for any school to admit students on the basis of ability, so schools for gifted learners do not exist. We have a long way to go to have any state provision of services for gifted children (called Exceptionally Able here). Most of our (scarce!) resources go to supporting those at the other end of the ability scale.

That's it in a nutshell really!
Originally Posted by Grinity
Thanks NCPMom - wow, so you skipped 'history' for 2 years? Did you study each subject for both years?
Grinity

Yes, I gave history up when I was about 14 - I had THE most boring history teacher in the world, and he totally put me off the subject, unfortunately. Also yes- you study all your GCSE subjects for the whole 2 years - and then when you do your 'A' Levels, you study just those 3 or 4 subjects for 2 more years.
Wow! I can see advantages and disadvantages.
Grinity
Posted By: Val Re: Freeman research vs A Nation Deceived etc. - 10/24/10 08:15 PM
Originally Posted by ColinsMum
Quote
The Target group was 70 children aged between five and 14, described as gifted by their parents, almost entirely without testing, all of whom had joined the National Association for Gifted Children. Each Target child was matched with two Control children
Quote
Of the whole sample, 170 children were at the 99th percentile of the Raven�s Matrices. Stanford-Binet IQs ranged from the 46 children with less than IQ120 to 18 children with above IQ160; 13 reached the Stanford-Binet test ceiling of 170 IQ.

Okay, I totally, completely do not understand these numbers. Maybe someone else can help.

Her data was collected in 1974, when the population of the UK was around 56 million. Of that number, probably around 11 million were kids under 18 (basing this on current numbers of 14.1 million 20 and under and making an estimate).

An IQ of 170 on an SB occurs once in 652,600 people. So finding 13 kids with that IQ means she enrolled more than 75% of this total population from all over the UK. Really? Wow. Are there any other people here who've identified and consented 75% of a national population of people with some very rare and non-obvious characteristic? If so, please tell me your secret. See arithmetic at end.

Yet I think her sample size was only 210, so she should have been surprised to find even one kid with an IQ of 145, let alone 170, even among a self-selected high IQ group.

More confusing is that she only had 46 people with IQs under 120. This is only 22% of her total population, yet an IQ of at least 120 occurs in 1 out of every 11 people. At a minimum, nearly everyone in her random-kids group (70 kids, right?) should have had an IQ below 120, yet even if all 46 were in the non-gifted group, they'd only comprise about 2/3 of it.

Bottom line: this data smells to me. It's puzzling why the reviewers didn't pick this up.

Maths:

13*652,600 = ~8.5 million
8.5 million/11 million = 0.77
0.77 * 100 = 77%
Quuuuiiite.

I forget which version of the SB it was she was using, though - if it was something with (then) outdated norms, the Flynn effect would account for some of it.

I'd actually also be interested to hear from someone qualified to give IQ tests what they think of the snippets of her testing children that appear in the Channel 4 "Child Genius" programs. At this distance I forget the details, but there were at least one or two occasions when she gushed that a child's answer (e.g. to a vocabulary definition question) was perfect when it seemed far from perfect to (admittedly utterly amateur) me - e.g., IIRR, she asked for similarities and differences between X and Y and the child gave only similarities. I did just wonder whether she might be systematically over-scoring the children she tested - which would also go some way to explaining these results (particularly if her own biases also influenced *which* children she over-scored most). One would hope this couldn't happen, and I'm certainly not asserting that it did; but it does seem clear that there's at least a bit of room for interpretation in which one relies on the tester's skill and lack of bias (indeed, were it not so, maybe one wouldn't need to be a qualified tester to administer the tests).
Having agreed (again) that the data smells, let me nevertheless point out one place where I think it doesn't smell quite as badly as you say.
Originally Posted by Val
More confusing is that she only had 46 people with IQs under 120. This is only 22% of her total population, yet an IQ of at least 120 occurs in 1 out of every 11 people. At a minimum, nearly everyone in her random-kids group (70 kids, right?) should have had an IQ below 120, yet even if all 46 were in the non-gifted group, they'd only comprise about 2/3 of it.

The ones you're calling random kids were not random; they were matched for SE level and school class with children who were parentally identified as highly-gifted. Factor in that IQ is not (and was not) nearly as socially acceptable a topic in the UK as in the US - and "giftedness" as a phenomenon is identified much less - it wouldn't be surprising if the children whose parents identified them as gifted, against this social pressure, tended to be quite extreme. Then factor in that IQ is correlated with SE level and with educational background, and it's not surprising that children from the same background and schools as the identified-gifted children were themselves far from average. If in addition to all this the numbers were on a test with outdated norms, that might do the rest for the paucity of children with IQs under 120. I don't see how to stretch this argument to account for the crazy numbers she has at the very top, or for the apparent ability to match raw scores on Raven's.
Posted By: Val Re: Freeman research vs A Nation Deceived etc. - 10/24/10 10:01 PM
Originally Posted by ColinsMum
The ones you're calling random kids were not random; they were matched for SE level and school class with children who were parentally identified as highly-gifted.

(Note: the kids were ID'ed as gifted rather than highly gifted.)

Okay, here are a couple things I see:

1. She never provides a definition of what constituted gifted in her study subjects. What kind of study doesn't define the central point, especially when all you need is a test score?

2. See quote:

Quote
The Second Control was taken at random from the class, culling a wide range of abilities from gifted to below average depending on the school class make-up. Some of the schools in the sample selected by ability so that in the triad matching, the random Second Control group child would more likely to be gifted, others were for all-comers so that the Second Control group child might be below average.

(By triad, she means the three kids in each individual comparison group.) So a major problem I have here is that she says that the control subjects weren't labelled as gifted. But, umm, if some were attending a school that selected for high ability kids, how could her controls possibly have avoided being labelled as, at a minimum, really smart?

Also, this is irrespective of socio-economic class. The paper says that subjects were matched by socio-economic class, not that they were all members of a single s-e class. So I stand by my assertion that the IQs of the second control group shouldn't have been so hugely skewed to the right. Remember, this was 1974 as well, and she herself admits that social mobility when the parents were growing up wouldn't have been the same as now (so, more high IQ people in the working classes).


Quote
Factor in that IQ is not (and was not) nearly as socially acceptable a topic in the UK as in the US - and "giftedness" as a phenomenon is identified much less - it wouldn't be surprising if the children whose parents identified them as gifted, against this social pressure, tended to be quite extreme.

See, for me this highlights another weakness of the paper. Namely, she's making you guess stuff that should have been spelled out.

One more thing:

Quote
Most subjects with an exceptionally high IQ, whether labelled or unlabelled as gifted, did much better in life then those with an average score...

Is it me, or does this statement undermine her anecdotes about some gifties becoming janitors or not using their PhDs?
Originally Posted by Val
(Note: the kids were ID'ed as gifted rather than highly gifted.
Well, she's not using either as a technical term, and she's inconsistent, and the children were identified without having been tested, so it's unclear (as you say).

Originally Posted by Val
2. See quote:

Quote
The Second Control was taken at random from the class, culling a wide range of abilities from gifted to below average depending on the school class make-up. Some of the schools in the sample selected by ability so that in the triad matching, the random Second Control group child would more likely to be gifted, others were for all-comers so that the Second Control group child might be below average.

(By triad, she means the three kids in each individual comparison group.) So a major problem I have here is that she says that the control subjects weren't labelled as gifted. But, umm, if some were attending a school that selected for high ability kids, how could her controls possibly have avoided being labelled as, at a minimum, really smart?

Also, this is irrespective of socio-economic class. The paper says that subjects were matched by socio-economic class, not that they were all members of a single s-e class. So I stand by my assertion that the IQs of the second control group shouldn't have been so hugely skewed to the right.
Your quote appears to contradict this one, which is what I was remembering - indeed I quoted it in the very first post of this thread:
Originally Posted by Freeman
Each Target child was matched with two Control children of the same sex, age and socio-economic level, sharing educational experience in the same school class.
This really isn't ambiguous, although maybe she contradicts herself elsewhere!

Originally Posted by Val
Remember, this was 1974 as well, and she herself admits that social mobility when the parents were growing up wouldn't have been the same as now (so, more high IQ people in the working classes).
I thought I remembered a lot of media fuss not very long ago about the fact that social mobility in the UK has not improved since then, actually. (Big political hot potato: one argument is that grammar schools were very good for social mobility.)

Originally Posted by Val
Quote
Most subjects with an exceptionally high IQ, whether labelled or unlabelled as gifted, did much better in life then those with an average score...

Is it me, or does this statement undermine her anecdotes about some gifties becoming janitors or not using their PhDs?
Does an unquantified "most" on a sentence with an undefined "better" undermine an anecdote? Who can say, what would it even mean? Let's not waste any more energy on this stuff.
Posted By: Val Re: Freeman research vs A Nation Deceived etc. - 10/28/10 04:25 PM
Originally Posted by ColinsMum
Let's not waste any more energy on this stuff.

I agree in some ways and not in others. What bothers me the most about this kind of "research" is that she has a high profile and presents herself as being an expert. Everything else aside, the fact that she didn't define "gifted" in that paper ("Here's a cutoff number, and here's how I got it for all these kids") renders the whole thing bogus to me.

Unfortunately, people listen to her, and therefore, from what I'm reading in this paper, some kids whose parents or teachers listen to her may not be told that they're cognitively gifted. Which means that they might go many years without understanding why they're so different from everyone else (and possibly blaming themselves for not fitting in).

Bottom line, I think that it's wrong to withhold information like this from people. It needs to be presented correctly and put in context, but still, kids have a right to know if people around them do.

Just my 2c.

Val




Posted By: Wren Re: Freeman research vs A Nation Deceived etc. - 10/31/10 02:00 PM
But what is the point?

Whether it is this Freeman research or the longitudianl research done with Hunter students, what makes a successful outcome for a child?

There are PG kids without acceleration that are highly successful, look at Sotomayer. And PG kids who have all kids of acceleration, challenges and make nothing of their lives.

After reading "The Element" by Ken Robinson, the ingredient that really matters, besides hard work, is passion. Without the passion and drive, you just won't be notable. Or take advantage of opportunities, according to him.

So when I see research that clearly defines "what exactly was the defining factor or factors, I am interested.

Ren
Posting on this old thread since the URL of the paper it discusses has changed (and the thread is too old for editing the original entry to be permitted). Here is the new one, for the benefit of anyone still interested. I'm still mystified by the paper, and would still like to hear from anyone who thinks they can give an explanation for the anomalies I started this thread with!
Posted By: Val Re: Freeman research vs A Nation Deceived etc. - 03/02/11 06:40 PM
Hi CM,

I'm not really sure what you're looking for. I skimmed through the paper (read more closely in parts), and my impression is that her study is flawed at best. Her writing is garbled in places, too, and I felt that the introduction rambled and failed to make her purpose clear.

This paper had a lot of flaws that have pointed out in this thread. I still find it very hard to believe that she could have had so many children at or above the 99th percentile, especially that so many could have had IQs of at least 170. I don't get the control group structure at all (esp. the second "control" group). It seems messy.

I don't know much about the Stanford-Binet test, but this article says that the ceiling of the revision that was released in 1986 (they administered it in the early 90s, right?) had a ceiling of 148. So I'm confused about her assertion that the ceiling of the test was 170. Other versions seem to have ceilings that are much higher than 170. She should have made this clear.

In fact, in looking through the paper, I saw that it's heavy on generalizations and has no quantitative data, apart from the dubious reporting of IQs ("Most of them now say....", "Too many had dissipated....", "Others, though, felt that....). I didn't find a single table in the entire paper. She didn't provide a copy of her survey questionnaire (if one even existed) or her basis for making judgments about "success." There was no statistical analysis, no reporting of percentages of subject's answers to questions, no nothing. Just lots-of-people-felt-this-way stuff. The entire thing seems to be subjective.

The study seems to be something of an edumacation project to me. I've seen these before (as a reviewer and reader): they're characterized by failure to use rigorous or even semi-rigorous methods, failure to structure the study properly, failure to provide data, and an apparent willingness to cherry-pick information that supports an original hypothesis, rather than allowing data to drive conclusions.

HTH....

Val


Originally Posted by Val
I'm not really sure what you're looking for.
Really, an explanation as to how something that looked like a peer-reviewed journal could publish something that looks so bad. Explanations I can think of include:
- this isn't really a peer-reviewed journal in the sense I understand it (nobody critically read the paper ever, or they did but nobody forced the author to rewrite); or
- actually it isn't as bad as I think, e.g. because someone who worked in the field and understood the conventions of the field would be able to see things that to them are obvious explanations of the discrepancies reported here.

Of these the first seems the more likely, but maybe I'm wrong or maybe there's another explanation I haven't thought of. I'd just like to know!
Posted By: Val Re: Freeman research vs A Nation Deceived etc. - 03/02/11 08:06 PM
Originally Posted by ColinsMum
Really, an explanation as to how something that looked like a peer-reviewed journal could publish something that looks so bad. Explanations I can think of include:
- this isn't really a peer-reviewed journal in the sense I understand it (nobody critically read the paper ever, or they did but nobody forced the author to rewrite); or
- actually it isn't as bad as I think, e.g. because someone who worked in the field and understood the conventions of the field would be able to see things that to them are obvious explanations of the discrepancies reported here.

Of these the first seems the more likely, but maybe I'm wrong or maybe there's another explanation I haven't thought of. I'd just like to know!

FWIW, the site I pulled it from classified it as a magazine. The guidelines for authors indicated that everything is peer-reviewed, so at least one or two people read it and approved of it. From my perspective, this reflects poorly on the publication.

It's hard for me to see a legitimate way around the lack of things that I listed in my last message (study design, data, etc.). If people who work in this field were to argue that their conventions don't require any of those things, I'd question the validity of their research even more. Unfortunately, from what I've seen, standards in the education field can be very low ("can be," not "always;" obviously not making a blanket statement about everyone here).

Val
I am not at all surprised at the publication of something that seems so poorly vetted. This is why ONE publication on a finding is "interesting" and five begin to be "convincing."


Often the 'independent' reviewers are nowhere near as unbiased or detached as they theoretically ought to be (in a small field it is all the more potentially incestuous). It's possible that this was a soft-reviewed paper, or that the journal only managed to generate a single 'qualified' reviewer, who happened to be too busy to REALLY review the paper, or tossed it to a borderline-competent or green graduate student or post-doc. That happens-- even in big name journals like Analytical Chemistry or Neuroscience, occasionally you'll find a publication that is cringe-worthy.

I wouldn't assume that it's you and that if you had enough expertise it would look 'better' to you. Chances are good that it would be even more obvious how awful the methodology or data analysis actually is. wink

I agree with Val here in general terms. The social science disciplines all too often train people to look at correlation and assume causative linkages, and the physical sciences tend to train people to avoid that very natural human impulse at all costs. <SIGH>

As every physical scientist learns: the plural of anecdote is NOT 'data' in any sense of the term.

Therefore, the so-called conclusions drawn from cherry-picked anecdotes are little more than pet conjectures, because the experimental design is frequently biased to such an extreme that it isn't even POSSIBLE to call any of the variables truly dependent or independent. <shrug>

I know a lot of physical scientists who privately snark pretty openly about social (airquotes) ''scientists'' and experimental design or statistics...('oh, look, isn't that cute?? They tried to use a two-tailed analysis here... how sweet that they tried... too bad that they don't explain why they dumped four data points from each trial.... Hmmm...'). This is what gave birth something called The Journal of Irreproducible Results. Think of it as MAD magazine for geeks. wink

My own graduate group was going to do a study once for them, I recall. Head circumfrence against a number of other things-- caffeine consumption, IQ, shoe size, GPA, number of siblings... We got quite nice correlations by plotting inverses against one another... <giggling> And the nice thing is, we just invalidated anything that didn't fit-- oh, sure, sometimes we had to LOOK for a reason to disqualify a study participant. But most people have corrected vision or peircings, or eat meat, have a family member who is a republican/atheist/nudist or something... grin

In all seriousness, though, I am very wary of ANYTHING plotted against even a simple mathematical transform of another quantity. Those relationships are frequently artifacts if there isn't a clear mechanistic reason posited or known for why the relationship should exist in the first place.


Now, does social science make for thought-provoking and sometimes insightful reading? Of course. I'm not saying that there aren't usually some interesting perspectives stated. But I tend to view most of it with a pretty jaded and critical eye. Mistakes/oversights in the statistics or in the experimental design are things I look for immediately in judging how much stock I should put in the conclusions section. I always consider whether a paper seems to be a reasonable example of a rigorous investigation... or if it's more of an op-ed bit written by an expert in the field. MOST of the literature in this field is the latter, unfortunately. That's an observation as much as a criticism-- there are reasons why sampling is so hard here. The same thing is true in the medical literature for rare genetic conditions.

smile
Originally Posted by HowlerKarma
I am not at all surprised at the publication of something that seems so poorly vetted. This is why ONE publication on a finding is "interesting" and five begin to be "convincing."


The social science disciplines all too often train people to look at correlation and assume causative linkages, and the physical sciences tend to train people to avoid that very natural human impulse at all costs. <SIGH>
I particularly want to 'me2' these two points. I think they should be taught to every high school level student, along with the finer points of why the scientific method is so beloved by it's followers, even with all the flawed humans who use it.

When I read Freeman's book, I was struck by all the 'association - causation' assumptions. She sure did find plenty of bad parenting amoung parents of gifted kids back in the 1970, but she never gets that the parents themselves are most likely gifted and grew up in even worse circumstances! And how difficult to raise would a child have had to be to drive a parent to seek identification in England in the 1970?

Remember the story of all those Mom's of kids with autism who got told that they were 'refridgerator moms' who caused their kid's behavior because they were cold and detached. Nowadays the causation arrow points 180 degrees in the opposite direction - raising an autistic kid without support is seen to cause moms to become cold and detached.

Live and learn -
Grinity
As a social scientist - Ouch!

I agree that there are a lot of crappy papers being published - in all fields. I also agree that social scientists should have a better understanding of statistics. But condemning whole disiplines...
Sorry. There are good and critical researchers working in those fields. I've had the pleasure of working with some of them. I definitely don't intend to malign the people working in those disciplines as a whole. I do fault some of the training that they are given, however, since it seems to lead to misunderstandings in how the scientific method is supposed to work...


I definitely think there is value in being able to trend-spot or to think outside of the causation box that scientists tend to live inside, though. That's where new ideas come from.

It's just that too much social science research doesn't design experiments so as to allow for the hypothesis to be proven incorrect. (Yes, it's a problem in some science disciplines now, too, as I'm well aware- positive results get published and funded, and nothing else seems to matter, which doesn't serve the discipline very well.) It's possible that it is merely publication bias, but I don't really think so, having seen what ed researchers cook up in experimental design.


But then again, experience also suggests that the maxims about scientists lacking basic social skills is also generally more often true than not...

so there's that. whistle Hopefully there are exceptions to THAT, as well.

Again-- apologies to any of the good social scientists out there that I may have inadvertently offended. I really do have a great deal of respect for my friends and co-authors from the other side of the campus, I promise. wink
smile Its all good.

Actually I think training social scientists - and I have to be honest and say I'm still 'in training' - is where there is so much potential wasted! There seems to be so much time spent on indoctrination to specific disciplines that the higher purpose of developing real, quality, critical thinking skills is ignored.

I also agree that statistics are so poorly understood in society at large and in some disciplines that it gives me headache. Few things drive me as batty as trying to explain to someone that 97% of people being x doesn't meant EVERYONE is and so them quoting what they're Great-Aunt Batty wasn't x doesn't change a thing! (Insert icon of someone running screaming in circles waving arms madly!)

But all of that is completely off-topic isn't it. I haven't actually read the article in question at all, so I have no on-track comment to make. smile
© Gifted Issues Discussion Forum