Being Correct Isn’t Enough: The Value of Confidence-Based Assessment in Medical Education

To measure knowledge, we must measure a person’s degree of belief.

As educators, we all strive to transfer knowledge and help our learners achieve mastery on a topic or technique.  And as educators, we rely on pre- and post-test comparisons to assess the impact of the education we provide, as well as inform refinements of current activities and identify future education needs – all with the goal of closing gaps and improving patient care.  However, the commonly used multiple-choice question employed in almost all CME assessments provides a hidden loophole in the form of guessing.  Simply put, a correct answer that was randomly chosen is not an indicator of true knowledge. Traditional multiple-choice assessments are of limited use in our ongoing efforts to develop more effective educational programs that truly address learners’ needs, unless we take our assessment questions a step further.    

CBA Digs Deeper

Not all right answers are equally correct, and not all wrong answers are incorrect in the same way. Applying confidence-based assessment (CBA) acknowledges the spectrum of learner responses and provides a bridge from traditional assessment to a more complete picture of a learner’s understanding. When you incorporate CBA into assessments, the results will help you identify those learners who are guessing as well as the ones who have doubt in their understanding, creating additional categories of answers beyond simply correct or incorrect. In this way, CBA shines a spotlight on true educational impact.

The strength of CBA is related to measuring how completely a learner embraces their understanding. Learners who confidently know an answer do not need to guess, while those who are unsure of their knowledge will make a selection based on partial knowledge. To uncover and measure guessing and doubt, CBA gauges a learner’s confidence in their chosen answer. This is done by layering an additional confidence query using some version of the straightforward phrases “I’m sure,” “I think,” and “I’m guessing.” The additional insight is collected for each question as the test is completed, and the confidence information is integrated into the test results, leading to stronger statistical reliability of assessment data.

CBA Research

Understanding the nuances of clinician learning is crucial to the success of education and has an impact on the future performance of learners and the advantages of CBA are particularly valuable in the decision-rich field of medical education, where misplaced reliability can lead to serious consequences.

Research indicates that clinician learners’ realism about their level of knowledge has an effect on their behavior as practitioners. Overconfident practitioners – the “misinformed” learners identified by CBA – present a real risk to patients. These are the ones who may refuse outside opinion and bypass lab tests, possibly delivering an incorrect diagnosis and/or inappropriate treatment plan, with additional risk of complications. Under-confident practitioners – “uninformed” or “in-doubt” learners – may routinely order excessive medical tests in order to be certain of a diagnosis, potentially wasting their patients’ time as well as healthcare resources.

Testing that incorporates CBA provides a robust analysis of healthcare practitioner learning and confidence in what they know, allowing the education provider to direct special attention to learners who are guessing, who lack confidence in their knowledge (the uninformed or in doubt), or have high confidence but are incorrect (the misinformed). This depth of data will identify important unmet learning needs, providing valuable opportunities to improve future assessments that will raise the bar on medical education, future healthcare practitioner performance, and, ultimately, patient outcomes.