Right-- but the inherent difficulty in assessing the results in such studies are that it is effectively impossible to tease apart motivation as part and parcel of the resultant effectiveness (as measured).
So if I like the color yellow, for example, I might be more inclined to pay attention to a document which is printed on yellow paper. It might have very little to do with the content or the writing style.
Confounding variables. Now, excellent experimental design accommodates that by using larger samples and well-designed controls which are "sham" treated in one way or another. Fancy stats ensue-- but I'm a little suspicious of a study that never even bothers to ask that kind of question-- whether or no the study subjects' own perceptions and biases might be influencing the resultant data in some way, I mean.
This is what I find maddening-- while I agree, AEH, that such a construct perhaps logically SHOULD exist, it is still predicated on that particular "domain" (for lack of a better term) in measures of intelligence-- which are merely models, and probably proxies (and imperfect ones at that) of what it is that they attempt to codify.
It's also the case that in studies where test subjects have been repeatedly evaluated, their classifications CHANGE. This is also true of Myers-Briggs, btw; there is a reason I mentioned that one.
So whatever it is that this tool is identifying-- it's not stable. Maybe it's even somewhat random.
Is the trait for which it is a proxy even real?
Nobody seems to know for sure. Not really. Yes, scores on individual subtests of cognitive ability seem to be stable, but that doesn't mean that those subtests are measuring what we hope that they are measuring. Entirely possible that they actually represent something else, which also happens to be a stable characteristic.
The really interesting underlying issue in my estimation is that people's preferences and perceptions may be wildly inaccurate in the first place, and that a great many individuals may THINK that they "learn best" in one way-- but can be entirely wrong about that.
I've been amused by students who have steadfastly maintained that they "prefer auditory" learning, yet they can recall what they do on homework problems and have excellent recall of material in a course text, but almost no recall of video or live instruction.
I've learned to just ignore whatever-it-is that people tell me about their Myers-Briggs "type" as well-- because that is another one that simply isn't stable within serial evaluations even for a single individual. I know a number of people who really, really believe in it, however. They are otherwise fairly well-educated people, too. I just smile.
