I read the paper. My impression was that it was a poorly designed study and wasn't measuring what they claimed it did. Rather than testing the quality of a good lecturer, I think they just gave the students a subsection of an IQ test.
The students sat at a computer individually and listened to a video for 65 seconds. It was about why calico cats are almost always female. Half the students listened to a good speaker, half to a not-so-good speaker. Content was the same.
Then they rated the speaker. After that, they answered 30 trivia questions as a distraction. Finally, they were given 5 minutes to write down everything they could remember about the topic. Surprise, surprise: performance didn't differ between the two groups. In a second experiment, they let people review the transcript and still got the same results (seems this particular outcome would be expected).
IMO, given that the content of the talks was the same, they were pretty much testing memory and have proven what has been proven umpteen times: the overall performance of two groups of people selected at random will be the same.
It seems to me that they missed an obvious and very important point, which is the content of the lecture. IMO, they should have compared FOUR lectures: one that flowed and made sense and one that was poorly conceived, both delivered by the two different speakers. But even so, there is still a large element of IQ testing there.
So what they really should have done was use students with the same IQs or with the same scores on digit span or another test that approximates ability for the task they were asked to do.
Okay, enough said. Personally, I'm not impressed with the design of this study.
Last edited by Val; 06/03/13 08:32 AM. Reason: More detail added
