Your point's very interesting, MumOfThree, and I didn't comment on it because I started down a trail of thinking about what you said and didn't get to a good resting point! My first intuition was: "I dare say in practice you're right, because it's so difficult to control for confounding factors, but I *think* that if the studies in question were done properly so as to show causation, not just correlation, your argument would not apply (or not exactly...)."
But then I started to actually do the mathematics and came to some surprising and amusing conclusions... And confused myself utterly, so let's see who else I can confuse :-) :-)
Let's consider a simplified situation. Suppose that we had uncontroversial, cast-iron evidence that
(a) using acrylic nappies raises IQ by 5 points ("acrylic" for short)
(b) giving a daily dose of cocaine raises IQ by 5 points ("cocaine" for short).
For causation to have been shown, which we're assuming, it has to be the case that, if you take groups of families with babies who are equally likely to have any characteristic known or suspected to affect IQ, except that one group uses acrylic and the other doesn't, then the acrylic group has babies with IQs on average 5 points higher. Similarly for cocaine. If the studies haven't shown this - if they haven't controlled for all plausible confounders - then by definition the studies are not sound. We're assuming they are.
Now, which of these connections, acrylic or cocaine, was suspected first? Let's suppose that the news about the possible benefit of acrylic broke first. For this investigation, it will suffice to imagine that, to everyone's surprise, the situation is very simple: every baby's IQ is raised by exactly 5 points compared to where it would otherwise have been expected to be, provided the baby wears an acrylic nappy. This is so convincing that noone questions it any more...
At this stage, nobody gives their baby cocaine; that wonder-factor hadn't yet been suspected. Later, giving your baby cocaine becomes fashionable and people think about studying it. Now, any study of cocaine that doesn't control for acrylic is by definition unsound. IOW, by assumption, we have studies that show that, even when the cocaine-using group and the non-cocaine using group have the same proportion of acrylic-users, the cocaine-using group has higher IQ. Is this consistent with there being no additional benefit of using cocaine if you already use acrylic?
Well, yes: it just has to be the case that there's a larger than 5 point effect on the non-acrylics, to account for the 0 point effect on the acrylics. Suppose for simplicity that 50% of the population uses acrylic at this point (and that the study's groups are representative in this respect). If non-acrylic + cocaine gives a 10 point advantage compared to neither, and acrylic + cocaine gives only the 5 point advantage they already had for acrylic, we're done: average IQ gain of 5 points for cocaine will be what we see when comparing the study groups, and will be what holds over the whole population.
Note the strange situation we've cooked up. Suppose the no-acrylic no-cocaine baseline IQ is 100. We've got acrylic no-cocaine and acrylic + cocaine both at 105, and we've got no-acrylic + cocaine at 110. Just so long as hardly anyone uses cocaine (the situation that pertained when the acrylic research was done) both acrylic and cocaine, separately, are genuinely causing a 5 point IQ increase averaged across the whole population, as stipulated: but the way to give your child the highest possible IQ is to *not* use acrylic but use cocaine.
As more and more people start using cocaine, the average benefit of using acrylic diminishes, until when 50% of people are using cocaine, the average effect of using acrylic is 0, and when 100% of people are using cocaine, we find that using acrylic (quite genuinely!) causes an average of a 5 point drop in your baby's IQ.
Now, as laid out, because the cocaine study is controlling for acrylic, this strange situation will, hopefully, be noticed and remarked on. (But maybe not - if all they do is to make sure the cocaine and no-cocaine groups have the same proportion of acrylic users, and they don't study the acrylic and non-acrylic users separately (maybe they haven't the power) they will still have controlled for acrylic, but they won't spot this effect.) But it illustrates the difficulty in investigating multi-factorial effects, and also the difficulty of applying research in a different time or place to where it was done.
If you consider a slightly different scenario, in which the same reality holds, but the research pattern is different, it could happen like this:
- 50% of people use acrylic, but only one group of researchers suspects it might matter. They do a study.
- 0.001% of people use cocaine, and only one group of researchers (not the same ones) think it might matter. They do a study.
Since the researchers are working independently and in ignorance of one another's work, neither controls for the other's factor. (Both control for all other known confounders, but that need not concern us: let us suppose that neither acrylic use nor cocaine use is correlated with any other confounder, nor with one another.) In our hypothesised reality, both studies will, quite rightly, find that their factor causes an average 5 point increase in IQ. And then maybe people like us who read both studies will come along and wonder whether or not you get a 10 point increase by doing both...
ETA: and to bring us back to reality, I wonder whether the study someone mentioned on playing string instruments controlled for breastfeeding, separate from the confounders of maternal IQ and eduction? It would be amusing, would it not, if playing a string instrument raises your IQ substantially but only if you weren't breastfed, so that actually, the optimal strategy for raising your baby's IQ is to formula feed and send them to Suzuki classes?
Last edited by ColinsMum; 08/22/11 04:26 AM.
