That's a GREAT explanation of good MCQ validation and monitoring, Colinsmum.
I just didn't have classes large enough that I ever felt that it was statistically feasible to validate things that well. I had to rely upon having a key and my knowledge of which students were in which quintile in order to validate individual questions-- but one thing that I really liked was that I had the evidence right in front of me when that top quintile (or maybe top two quintiles) were approaching a question in a way that I hadn't anticipated.
At least since they weren't scantronned, though, I had the ability to modify credit awarding
on the fly to account for my human errors in constructing assessment items.
