Evaluating the effectiveness of a mock oral educational program

Background To obtain board certification, the American Board of Surgery requires graduates of general surgery training programs to pass both the written qualifying examination (QE) and the oral certifying examination (CE). In 2015, the pass rates for the QE and CE were 80% and 77%, respectively. In the 2011-2012 academic year, the University of Wisconsin instituted a mandatory, faculty-led, monthly CE preparation educational program (CE prep) as a supplement to their existing annual mock oral examination. We hypothesized that the implementation of these sessions would improve the first-time pass rate for residents taking the ABS CE at our institution. Secondary outcomes studied were QE pass rate, correlation with American Board of Surgery In-Training Examination (ABSITE) and mock oral examination scores, cost, and type of study materials used, perception of examination difficulty, and applicant preparedness. Methods A sixteen question survey was sent to 57 of 59 residents who attended the University of Wisconsin between the years of 2007 and 2015. Email addresses for two former residents could not be located. De-identified data for the ABSITE and first-time pass rates for the QE and CE examination were retrospectively collected and analyzed along with survey results. Statistical analysis was performed using SPSS version 22 (IBM Corp., Armonk, NY). P values < 0.05 were considered significant. Results Survey response rate was 77.2%. Of the residents who have attempted the CE, first-time pass rate was 76.0% (19 of 25) before the implementation of the formal CE Prep and 100% (22 of 22) after (P = 0.025). Absolute ABSITE score, and mock oral annual examination grades were significantly improved after the CE Prep was initiated (P values < 0.001 and 0.003, respectively), however, ABSITE percentile was not significantly different (P = 0.415). ABSITE raw score and percentile, as well as mock oral annual examination scores were significantly associated with passing the QE (0.032, 0.027, and 0.020, respectively), whereas mock oral annual examination scores alone were associated with passing the CE (P = 0.001). Survey results showed that residents perceived the CE to be easier than the annual mock oral after the institution of the CE prep course (P = 0.036), however, there was no difference in their perception of preparedness. Overall, applicants felt extremely prepared for the CE (4.70 ± 0.5, Likert scale 1-5). Conclusions Formal educational programs instituted during residency can improve resident performance on the ABS certifying examination. The institution of a formal, faculty-led monthly CE preparation educational program at the University of Wisconsin has significantly improved the first-time pass rate for the ABS CE. Mock oral annual examination scores were also significantly improved. Furthermore, ABSITE scores correlate with QE pass rates, and mock oral annual examination scores correlate with pass rates for both QE and CE.