Access to High Level Clinical Protocols do not influence trainee knowledge about mechanical ventilation

Abstract

Context Clinical protocols are associated with improved patient outcomes; however, they may negatively affect medical education by removing trainees from clinical decision making.

Objective To study the relationship between critical care training with mechanical ventilation protocols and subsequent knowledge about ventilator management.

Design, Setting, and Participants A retrospective cohort equivalence study, linking a national survey of mechanical ventilation protocol availability in accredited US pulmonary and critical care fellowship programs with knowledge about mechanical ventilation among first-time examinees of the American Board of Internal Medicine (ABIM) Critical Care Medicine Certification Examination in 2008 and 2009. Exposure to protocols was defined as high intensity if an examinee's training intensive care unit had 2 or more protocols for at least 3 years and as low intensity if 0 or 1 protocol.

Main Outcome Measures Knowledge, measured by performance on examination questions specific to mechanical ventilation management, calculated as a mechanical ventilation score using item response theory. The score is standardized to a mean (SD) of 500 (100), and a clinically important difference is defined as 25. Variables included in adjusted analyses were birth country, residency training country, and overall first-attempt score on the ABIM Internal Medicine Certification Examination.

Results Ninety of 129 programs (70%) responded to the survey. Seventy-seven programs (86%) had protocols for ventilation liberation, 66 (73%) for sedation management, and 54 (60%) for lung-protective ventilation at the time of the survey. Eighty-eight (98%) of these programs had trainees who completed the ABIM Critical Care Medicine Certification Examination, totaling 553 examinees. Of these 88 programs, 27 (31%) had 0 protocols, 19 (22%) had 1 protocol, 24 (27%) had 2 protocols, and 18 (20%) had 3 protocols for at least 3 years. Forty-two programs (48%) were classified as high intensity and 46 (52%) as low intensity, with 304 trainees (55%) and 249 trainees (45%), respectively. In bivariable analysis, no difference in mean scores was observed in high-intensity (497; 95% CI, 486-507) vs low-intensity programs (497; 95% CI, 485-509). Mean difference was 0 (95% CI, –16 to 16), with a positive value indicating a higher score in the high-intensity group. In multivariable analyses, no association of training was observed in a high-intensity program with mechanical ventilation score (adjusted mean difference, −5.36; 95% CI, –20.7 to 10.0).

Conclusion Among first-time ABIM Critical Care Medicine Certification Examination examinees, training in a high-intensity ventilator protocol environment compared with a low-intensity environment was not associated with worse performance on examination questions about mechanical ventilation management.

via jama.ama-assn.org

Just read this in this week's special issue on medical education in JAMA. This is an interesting study while the specifics of this study will still need to be discussed (something I plan to write more in myblog here), I think for now it's fair that it is an interesting finding from the larger perspective that existence of protocols do not necessarily influence someone's decision making skills. At UC, we are developing our own simulations and I thought this would be a good study to discuss further.