TY - JOUR
T1 - Validity Evidence for a Serious Game to Assess Performance on Critical Pediatric Emergency Medicine Scenarios
AU - Gerard, James M.
AU - Scalzo, Anthony J.
AU - Borgman, Matthew A.
AU - Watson, Christopher M.
AU - Byrnes, Chelsie E.
AU - Chang, Todd P.
AU - Auerbach, Marc
AU - Kessler, David O.
AU - Feldman, Brian L.
AU - Payne, Brian S.
AU - Nibras, Sohail
AU - Chokshi, Riti K.
AU - Lopreiato, Joseph O.
N1 - Funding Information:
Supported by the Office of Naval Research.
Publisher Copyright:
© 2018 by the Society for Simulation in Healthcare.
PY - 2018/6/1
Y1 - 2018/6/1
N2 - Introduction We developed a first-person serious game, PediatricSim, to teach and assess performances on seven critical pediatric scenarios (anaphylaxis, bronchiolitis, diabetic ketoacidosis, respiratory failure, seizure, septic shock, and supraventricular tachycardia). In the game, players are placed in the role of a code leader and direct patient management by selecting from various assessment and treatment options. The objective of this study was to obtain supportive validity evidence for the PediatricSim game scores. Methods Game content was developed by 11 subject matter experts and followed the American Heart Association's 2011 Pediatric Advanced Life Support Provider Manual and other authoritative references. Sixty subjects with three different levels of experience were enrolled to play the game. Before game play, subjects completed a 40-item written pretest of knowledge. Game scores were compared between subject groups using scoring rubrics developed for the scenarios. Validity evidence was established and interpreted according to Messick's framework. Results Content validity was supported by a game development process that involved expert experience, focused literature review, and pilot testing. Subjects rated the game favorably for engagement, realism, and educational value. Interrater agreement on game scoring was excellent (intraclass correlation coefficient = 0.91, 95% confidence interval = 0.89-0.9). Game scores were higher for attendings followed by residents then medical students (Pc < 0.01) with large effect sizes (1.6-4.4) for each comparison. There was a very strong, positive correlation between game and written test scores (r = 0.84, P < 0.01). Conclusions These findings contribute validity evidence for PediatricSim game scores to assess knowledge of pediatric emergency medicine resuscitation.
AB - Introduction We developed a first-person serious game, PediatricSim, to teach and assess performances on seven critical pediatric scenarios (anaphylaxis, bronchiolitis, diabetic ketoacidosis, respiratory failure, seizure, septic shock, and supraventricular tachycardia). In the game, players are placed in the role of a code leader and direct patient management by selecting from various assessment and treatment options. The objective of this study was to obtain supportive validity evidence for the PediatricSim game scores. Methods Game content was developed by 11 subject matter experts and followed the American Heart Association's 2011 Pediatric Advanced Life Support Provider Manual and other authoritative references. Sixty subjects with three different levels of experience were enrolled to play the game. Before game play, subjects completed a 40-item written pretest of knowledge. Game scores were compared between subject groups using scoring rubrics developed for the scenarios. Validity evidence was established and interpreted according to Messick's framework. Results Content validity was supported by a game development process that involved expert experience, focused literature review, and pilot testing. Subjects rated the game favorably for engagement, realism, and educational value. Interrater agreement on game scoring was excellent (intraclass correlation coefficient = 0.91, 95% confidence interval = 0.89-0.9). Game scores were higher for attendings followed by residents then medical students (Pc < 0.01) with large effect sizes (1.6-4.4) for each comparison. There was a very strong, positive correlation between game and written test scores (r = 0.84, P < 0.01). Conclusions These findings contribute validity evidence for PediatricSim game scores to assess knowledge of pediatric emergency medicine resuscitation.
KW - Pediatric Advanced Life Support
KW - Serious game
KW - assessment
KW - simulation
UR - http://www.scopus.com/inward/record.url?scp=85048724880&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85048724880&partnerID=8YFLogxK
U2 - 10.1097/SIH.0000000000000283
DO - 10.1097/SIH.0000000000000283
M3 - Article
C2 - 29377865
AN - SCOPUS:85048724880
SN - 1559-2332
VL - 13
SP - 168
EP - 180
JO - Simulation in Healthcare
JF - Simulation in Healthcare
IS - 3
ER -