You are here

Article Text

Article menu

Download PDFPDF

Original Article
Simulation training improves ability to manage medical emergencies
  1. Miriam Ruesseler1,
  2. Michael Weinlich2,
  3. Michael P Müller3,
  4. Christian Byhahn4,
  5. Ingo Marzi1,
  6. Felix Walcher1
  1. 1Department of Trauma Surgery, J.W. Goethe University Hospital, Frankfurt, Germany
  2. 2Med Con Team, CEO, Reutlingen, Germany
  3. 3Department of Anaesthesiology and Critical Care Medicine, University Hospital Carl Gustav Carus, University of Technology, Dresden, Germany
  4. 4Department of Anaesthesiology, Intensive Care Medicine and Pain Therapy, J.W. Goethe University Hospital, Frankfurt, Germany
  1. Correspondence to Miriam Ruesseler, Department of Trauma Surgery, J.W. Goethe University Hospital, Theodor-Stern-Kai 7, Frankfurt 60590, Germany; miri{at}mruesseler.de

Abstract

Objective In the case of an emergency, fast and structured patient management is crucial for a patient's outcome. Every physician and graduate medical student should possess basic knowledge of emergency care and the skills to manage common emergencies. This study determines the effect of a simulation-based curriculum in emergency medicine on students' abilities to manage emergency situations.

Methods A controlled, blinded educational trial of 44 final-year medical students was carried out at Frankfurt Medical School; 22 students completed the former curriculum as the control group and 22 the new curriculum as the intervention group. The intervention consists of simulation-based training with theoretical and simulation-based training sessions in realistic encounters based on the Basic Life Support (BLS), Advanced Cardiac Life Support (ACLS) and adapted Advanced Trauma Life Support (ATLS) training. Further common emergencies were integrated corresponding to the course objectives. All students faced a performance-based assessment in a 10 station Objective Structured Clinical Examination (OSCE) using checklist rating within a maximum of 4 months after completion of the intervention.

Results The intervention group performed significantly better at all of the 10 OSCE stations in the checklist rating (p<0.0001 to p=0.016).

Conclusions The simulation-based intervention offers a positively evaluated possibility to enhance students' skills in recognising and handling emergencies. Additional studies are required to measure the long-term retention of the acquired skills, as well as the effect of training in healthcare professionals.

  • Clinical assessment
  • education
  • effectiveness
  • resuscitation, training

https://doi.org/10.1136/emj.2009.074518

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    A fast and structured patient management algorithm is crucial for the outcome of acutely ill and injured patients. Society has a right to expect that all physicians possess the basic knowledge and skills to manage common emergencies.1 All students graduating from medical school should be capable of handling emergencies, as they are generally the first on scene as residents.

    One method to teach practical skills is through simulation. Simulation has been used to increase knowledge, to provide opportunities for deliberate and safe practice, and to shape the development of clinical skills in diverse specialities and for various skills.2–5 However, only few studies address the effect of simulation training regarding its effectiveness on practical skills in comparison with other teaching methods,2 especially in emergency medicine. Ten Eyck et al6 demonstrated in a randomised controlled trial the improved performance of fourth-year medical students in a simulation format on a multiple choice examination in comparison with case-based group discussions. However, they lack the measurement on students' practical skills. Steadman et al7 demonstrated superior effectiveness of simulation training in comparison to an interactive problem-based learning format, though the students were assessed in unique scenarios focused solely on dyspnoea and thereby missed the evaluation of effectiveness of simulation training in the broad range of medical emergencies and the different competencies needed.

    This study was undertaken to evaluate the effect of the simulation training on undergraduate medical students' skills managing emergencies. As a performance-based assessment method, the Objective Structured Clinical Examination (OSCE) was chosen. In a recent study, the feasibility, authenticity and high reliability of a 10 station OSCE in emergency medicine was demonstrated.8

    It is hypothesised that residents exposed to the new curriculum would demonstrate superior management skills compared to those who participated in the former curriculum.

    Methods

    Study design

    The present study had an intervention-control design of a standardised, simulation-based educational intervention designed to increase undergraduate medical students' skills in management of emergencies. The measurement was obtained within a maximum of 4 months after students completed the intervention period.

    Study population

    Study participants were final-year medical students at Frankfurt Medical School (Frankfurt, Germany). Due to changes in the curriculum of undergraduate medical education, there was the unique constellation of having final-year students who participated in two different education programs during the transition period from the former to the new curriculum depending on their start of medical studies in either October 2000 or April 2001.

    The intervention group (n=22) completed the new curriculum of medical education and the control group (n=22) completed the former medical training. The last year of both curricula consists of three 16 week electives in each of the following three disciplines: internal medicine, surgery and one elective subject of the student's own choosing. All students in their internal medicine or surgery elective at the J.W. Goethe University Hospital were invited to participate in this study. Twenty-two of 39 students in the control group, as well as 30 of 45 students in the intervention group consented to participate in this study. Seven students in the intervention group had to be excluded as they had participated in the intervention more than 4 months previously. Due to illness, there was one drop out in the intervention group prior to the beginning of measurement.

    As stated by the Ethics board of the medical faculty of J.W. Goethe University Hospital, Frankfurt, Germany, an ethical approval was not required for this study. The research and introduction of new educational methods is demanded in the new regulations on the licence to practice medicine in Germany and supported by the medical faculty. Nevertheless, all participants provided written informed consent to participate in this study and to be photographed and filmed during the study, prior to the beginning of the study.

    Study protocol

    Intervention and control groups completed their emergency medicine training and passed their assessments according to their curriculum (figure 1). They completed identical training in medical first aid and the same interdisciplinary lectures in their first and second semesters. The interdisciplinary lectures cover the main cardinal symptoms and their algorithm-based treatment and management, trauma management, aspects of team work and crew resource management. The control group had to participate in a minimum of three obligatory shifts in the emergency department. The intervention group had to successfully complete the 3 day standardised, simulation-based intervention.

    Figure 1
    Figure 1

    Curriculum emergency medicine. The figure demonstrates the curriculum for both groups from first to eighth clinical semester, corresponding to third to sixth year of undergraduate medical education in Germany. *Completed via multiple choice questionnaire.

    The intervention was designed to help students acquire and shape clinical skills needed to manage in-hospital and prehospital emergencies. The training program is based on 1 day Basic Life Support (BLS) and 2 day Advanced Cardiac Life Support (ACLS) provider training, in which 3 h Advanced Trauma Life Support (ATLS)-based training is integrated. Further common emergencies, for instance acute asthma, were added to the training concept corresponding to the course objectives. The intervention consists of short theoretical sessions to repeat the algorithms learned in the lectures and a series of standardised scenarios in realistic encounters. The simulation training was conducted at the Frankfurt Institute of Emergency Medicine and Simulation Training (Frankfurt, Germany)9 using mobile manikins (Resusci Anne Simulator, Laerdal Medical AS, Stavanger, Norway). The training centre offers various authentic scenario rooms, for example, an ambulance room for the realistic simulation of clinical events. The approximation to reality is one of the basic principles of the training centre, as the environment influences the efficiency of learning and the reproduction of information.10

    A maximum of six students was trained in each teaching session by one ACLS instructor. During the scenarios, one student performed as team leader, while the others performed the tasks directed by him/her. The students rotated between the positions so that each had several chances to act as team leader. As described in the ACLS guidelines and the course objectives in the National Medical Student Emergency Medicine Curriculum Guide,11 the students were expected to obtain a patient's history and physical examination focused on the key problem, request non-invasive monitoring, develop an evaluation and treatment plan, order medications and consider the further patient management. The scenarios are arranged to span a broad range of emergency situations, becoming increasingly challenging as the course progresses. After each scenario, a debriefing session with feedback and correction in a supportive environment followed as operational rule of the educational intervention.12 13

    Measurement

    In order to assess students' competencies to manage emergencies, the OSCE was used as performance-based assessment consisting of six complete emergency scenarios (three with standardised patients=SP,14 three with manikins using Resusci Anne Simulator), as well as four skills OSCE stations. The OSCE has been shown to be a valid and reliable assessment instrument for clinical competence in a comprehensive, consistent and structured manner.8 15–18 It combines the simulated reality of clinically relevant interactions, standardisation of testing situations and use of multiple observations by multiple examiners.14

    The stations were designed based on a blueprint representing the curricular goals and objectives of the new curriculum in emergency medicine.19 20 In the scenario stations, students had to perform complete management of an emergency from initial assessment to treatment plan and further considerations as he/she would do as physician in charge. For each of the scenarios, students had an actor to their side playing the role of a nurse for in-house emergencies and that of a paramedic for prehospital scenarios respectively. Students could order their assistance, for example, to perform blood pressure measurement, IV-line insertion or drug administration. In the skills OSCE stations, students had to perform single tasks, for example, preparation and insertion of an intraosseous access. A maximum of eight students participated in each OSCE circle. Based on the result of several pilot runs and the previous study, a time frame of 8 min was given for completing each scenario station, as well as 4 min for the skills OSCE stations.8

    The scenarios reflect real cases adapted to the objectives of the blueprint. They include a detailed description of SP's/manikin's portrayal of the patient, setting, vital parameters, information about the patient's history, as well as detailed further information for the SP, assistant and examiner.20 SP and assistant were extensively trained together in preparation for their roles in a reading and a half day practical training.21 22

    The students were scored during performance via checklist. The checklists were designed based on the learning objectives of the students' emergency medicine programme, and covered the abilities in emergency treatment expected by every doctor.20 23 The checklists were task-specific, but had an identical structure based on the ABCD-Algorithms taught and comprised rigorous step-by-step procedures.24 The checklists were reviewed for completeness and accuracy by an expert panel of ACLS instructors, emergency physicians and OSCE experts of other faculties, and were furthermore tested and re-evaluated in dry runs and a previous study. For the checklist items, a dichotomous scoring scale of not done/incorrectly and done/correct was imposed. The expert panel decided to total the items of each scenario station to an overall score of 25 points, and 12.5 points for the skills stations as they take place in half the time. The items of each scenario station were assigned to five categories (basic assessment, advanced assessment and initiation of therapy, taking actual and past medical history, patient's management, report and handover). Within the expert panel the weighting and scoring of each item was discussed and defined based on the importance and significance of the learning objective to student training and its importance in the scenario.

    The examiners were not part of the expert panel, nor were they involved in the scenario design or the checklist item weighting. They were blinded towards the item weighting. The examiners participated in a half day training session, where they received an introduction to the OSCE and the scenarios, and gained experience in the use of the checklists by scoring videos of the recent study. The participating examiners, SPs and assistants, as well as the students were blinded towards the study protocol; examiners were furthermore blinded towards students' group membership. None of the examiners was involved in the intervention group simulation training.

    A review session followed each OSCE circle conducted with all participants (students, examiners, SPs and assistants) for debriefing and feedback.

    Data analysis

    A sample size analysis was performed prior to the study using Power and Sample Size Calculations (Version 2.1.31). Based on the results of pilot runs conducted prior to the study, a difference in population mean (δ) of 5.0 with a SD 3.0 (σ) is assumed. Using a p value of 0.001 and a power of 0.9, a sample size of 17 per group was estimated.

    The statistical analysis was performed using SPSS 12.0 for the checklist and Microsoft Excel for the evaluation. Once Gaussian distribution of the data was verified, values were presented as mean±SD. Checklist score reliability was estimated using Cronbach's Alpha (α) coefficient as recommended by Streiner and Norman.25 26 Interventions versus control group differences were analysed using independent samples Student t tests.

    Results

    The OSCE circle could be run without any delay. All 44 students enrolled in this study consented to participate and completed the entire study. All intervention group students (n=13 men, n=9 women) successfully completed the entire programme. None of the control group students (n=12 men, n=10 women) participated in any of the intervention group training.

    The hypothesis of the study, that simulation-based training is superior to theory-based curriculum, was clearly supported by the checklist results in all OSCE stations as presented in table 1. In particular, table 1 shows that the mean scores of all OSCE stations were significantly higher in the intervention group as compared to the control group (p<0.0001 to p<0.016). The reliability coefficient, expressed as Cronbach's Alpha internal consistency reliability, indicates a high internal consistency. Intervention and control groups scored best at the ‘One Rescuer CPR’ scenario station, with the intervention group getting 90% of the maximum points, compared to 62% for the control group. Both groups scored worst at the ‘Trauma’ scenario station, with the intervention group getting 76% of maximum score, compared to 52% for the control group.

    View this table:
    Table 1

    Checklist descriptive statistics and reliabilities for intervention group and control

    For further analysis, the checklist items were assigned to categories based on the algorithms (ABCD). Figures 2A–D demonstrate exemplified analysis of four scenario stations in per cent of maximum score for intervention and control groups. This comparison demonstrates the highest deficits of the control group in initial primary survey, secondary survey with additional diagnostics and medical/drug therapy.

    Figure 2
    Figure 2

    Results of the item categories in scenario station. (A) Acute Coronary Syndrome. (B) Stroke. (C) Trauma. (D) Prehospital CPR with AED. Given are mean scores in % of the total score for intervention (dark grey) and control group (light grey) for each item category. AED, automatic external defibrillator; CPR, cardiopulmonary resuscitation.

    The evaluation data from all students participating in the new curriculum are uniformly positive. Seventy-four per cent rated the overall program as excellent, 26% as good. Ninety-seven per cent rated the program to be expedient and reasonable for their future medical occupation.

    All participants of the control group were offered the opportunity to participate in ACLS training. Nineteen students accepted this offer and successfully became ACLS providers.

    Discussion

    A main goal of medical education is to train students to be able to cope with real-life situations, especially in emergency medicine, where highly skilled performance is crucial.27 In this study, it was demonstrated that practical training does have a huge impact on abilities in emergency medicine. Previous studies with internal medicine residents have demonstrated the beneficial effects of simulation training in ACLS.3 28 This study demonstrates that at earlier stages of medical education, standardised, algorithm-based simulation training offers a good possibility to help students acquire and improve practical skills in emergency medicine in a realistic, yet safe environment.

    However, the design and workflow of the OSCE scenarios was in parts similar to the one in intervention training, and, thus, intervention scores may be inflated by a training effect. On the other hand, the control group participated in emergency department shifts with approximately 20 supervised patient contacts per day following the same workflow as used in the intervention group, diminishing the inflation of scores.

    One responsibility as an educator is to design or enhance assessment instruments that will predict performance in real-life situations.27 Even though the OSCE approximates a realistic encounter, the result cannot be simply generalised directly to clinical practice. Hereby, further studies about the effect of students' abilities to manage emergencies on patient outcome are required. However, this does not diminish the pronounced impact of the standardised training in student education, as its beneficial effects in ACLS training have already been proven in clinical residents.3

    The hypothesis that the educational intervention does significantly improve student competencies in handling emergencies could be proven by the data presented. The implementation of the simulation training in the undergraduate medical curriculum enhances student skills in a very positively evaluated manner. The evaluations demonstrate the high level of enjoyment and motivation students gained from the simulation-based training. They prefer to acquire skills in a realistic encounter. Furthermore, they prefer to be assessed in a realistic encounter as it helps them to determine whether they are ready to cope with real-life situations.

    Both groups were in their final year of studies, where students work as electives in the major medical subjects and have no more specific practical training, especially not in emergency medicine. This means the competencies assessed in this study correspond to the competencies students will possess as graduated physicians. With these significant results, this study confirms the necessity and effectiveness of the implementation of simulation training in emergency medicine.

    Conclusion

    This study demonstrates the huge impact of standardised training on the development of practical competencies in emergency medicine. Based on the results of the current study, the complete manpower intensive simulation training was introduced into the undergraduate curriculum, resulting in training of over 300 students per year.

    Acknowledgments

    We would like to thank the clinicians, paramedics and course instructors who contributed to the construction and evaluation of scenarios and marking schemes. We would like to thank Dr. Dirk Henrich and Dr. Sebastian Wutzler for their statistical advice. Our special thanks go to the students, examiners, standardised patients and assistants who took part in this study.

    References

    1. ↵
      1. Burdick WP,
      2. Jouriles NJ,
      3. D'Onofrio G,
      4. et al
      . Emergency medicine in undergraduate education. SAEM Education Committee, Undergraduate Subcommittee, Society for Academic Emergency Medicine. Acad Emerg Med 1998;5:1105–10.
      OpenUrlPubMedWeb of Science
    2. ↵
      1. Issenberg SB,
      2. McGaghie WC,
      3. Petrusa ER,
      4. et al
      . Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach 2005;27:10–28.
      OpenUrlCrossRefPubMedWeb of Science
    3. ↵
      1. Wayne DB,
      2. Butter J,
      3. Siddall VJ,
      4. et al
      . Simulation-based training of internal medicine residents in advanced cardiac life support protocols: a randomized trial. Teach Learn Med 2005;17:202–8.
      OpenUrlPubMedWeb of Science
    4. ↵
      1. Issenberg SB,
      2. Pringle S,
      3. Harden RM,
      4. et al
      . Adoption and integration of simulation-based learning technologies into the curriculum of a UK Undergraduate Education Programme. Med Educ 2003;37(Suppl 1):42–9.
      OpenUrlCrossRefPubMedWeb of Science
    5. ↵
      1. Issenberg SB,
      2. McGaghie WC,
      3. Hart IR,
      4. et al
      . Simulation technology for health care professional skills training and assessment. JAMA 1999;282:861–6.
      OpenUrlCrossRefPubMedWeb of Science
    6. ↵
      1. Ten Eyck RP,
      2. Tews M,
      3. Ballester JM
      . Improved medical student satisfaction and test performance with a simulation-based emergency medicine curriculum: a randomized controlled trial. Ann Emerg Med 2009;54:684–91.
      OpenUrlCrossRefPubMedWeb of Science
    7. ↵
      1. Steadman RH,
      2. Coates WC,
      3. Huang YM,
      4. et al
      . Simulation-based training is superior to problem-based learning for the acquisition of critical assessment and management skills. Crit Care Med 2006;34:151–7.
      OpenUrlCrossRefPubMedWeb of Science
    8. ↵
      1. Ruesseler M,
      2. Weinlich M,
      3. Byhahn C,
      4. et al
      . Increased authenticity in practical assessment using emergency case OSCE stations. Adv Health Sci Educ Theory Pract 2009;15:81–95.
      OpenUrlCrossRefPubMed
    9. ↵
      1. Anonymous
      . Frankfurt institute for emergency medicine and simulation training. http://www.finest-online.org (accessed 28 Feb 2008).
    10. ↵
      1. Godden D,
      2. Baddeley A
      . Context dependent memory in two natural environments. On land and under water. Br J Psychol 1975;66:325–31.
      OpenUrlCrossRefWeb of Science
    11. ↵
      1. Manthey D,
      2. Coates WC,
      3. Ander D,
      4. et al
      . Report of the task force on national fourth year medical student emergency medicine curriculum guide. Ann Emerg Med 2006;47:1–7.
      OpenUrlCrossRefPubMedWeb of Science
    12. ↵
      1. Ende J
      . Feedback in clinical medical education. JAMA 1983;250:777–81.
      OpenUrlCrossRefPubMedWeb of Science
    13. ↵
      1. Branch WT Jr.,
      2. Paranjape A
      . Feedback and reflection: teaching methods for clinical settings. Acad Med 2002;77:1185–8.
      OpenUrlCrossRefPubMedWeb of Science
    14. ↵
      1. van der Vleuten C,
      2. Swanson D
      . Assessment of clinical skills with standardized patients: state of the art. Teach Learn Med 1990;2:58–76.
      OpenUrlCrossRef
    15. ↵
      1. Harden RM,
      2. Gleeson FA
      . Assessment of clinical competence using an objective structured clinical examination (OSCE). Med Educ 1979;13:41–54.
      OpenUrlPubMedWeb of Science
    16. ↵
      1. Hodges B,
      2. Regehr G,
      3. Hanson M,
      4. et al
      . Validation of an objective structured clinical examination in psychiatry. Acad Med 1998;73:910–12.
      OpenUrlCrossRefPubMedWeb of Science
    17. ↵
      1. Crossley J,
      2. Humphris G,
      3. Jolly B
      . Assessing health professionals. Med Educ 2002;36:800–4.
      OpenUrlCrossRefPubMedWeb of Science
    18. ↵
      1. Hodges B
      . Validity and the OSCE. Med Teach 2003;25:250–4.
      OpenUrlCrossRefPubMedWeb of Science
    19. ↵
      1. Newble D
      . Techniques for measuring clinical competence: objective structured clinical examinations. Med Educ 2004;38:199–203.
      OpenUrlCrossRefPubMedWeb of Science
    20. ↵
      1. Hodges B,
      2. Hanson M,
      3. McNaughton N,
      4. et al
      . Creating, monitoring, and improving a psychiatry OSCE: a guide for faculty. Acad Psychiatry 2002;26:134–61.
      OpenUrlCrossRefPubMedWeb of Science
    21. ↵
      1. Boursicot K,
      2. Roberts T
      . How to set up an OSCE. Clin Teach 2005;2:16–20.
      OpenUrlCrossRef
    22. ↵
      1. Battles JB,
      2. Wilkinson SL,
      3. Lee SJ
      . Using standardised patients in an objective structured clinical examination as a patient safety tool. Qual Saf Health Care 2004;13(Suppl 1):46–50.
      OpenUrlAbstract/FREE Full Text
    23. ↵
      1. Selby C,
      2. Osman L,
      3. Davis M,
      4. et al
      . Set up and run an objective structured clinical exam. BMJ 1995;310:1187–90.
      OpenUrlFREE Full Text
    24. ↵
      The checklists development checklist. http://www.wmich.edu/evalctr/checklists/cdc.htm (accessed 28 Feb 2008).
    25. ↵
      1. Streiner DL,
      2. Norman GR
      1. Streiner DL,
      2. Norman GR
      . Selecting the items. In: Streiner DL, Norman GR, eds. Health measurement scales: a practical guide to their development and use. 3rd edn. Oxford: Oxford University Press, 2003:61–79.
    26. ↵
      1. Streiner DL,
      2. Norman G
      1. Streiner DL,
      2. Norman GR
      . Reliability. In: Streiner DL, Norman G, eds. Health measurement scales: a practical guide to their development and use. Oxford: Oxford University Press, 2003:126–52.
    27. ↵
      1. Lunenfeld E,
      2. Weinreb B,
      3. Lavi Y,
      4. et al
      . Assessment of emergency medicine: a comparison of an experimental objective structured clinical examination with a practical examination. Med Educ 1991;25:38–44.
      OpenUrlCrossRefPubMedWeb of Science
    28. ↵
      1. Wayne DB,
      2. Siddall VJ,
      3. Butter J,
      4. et al
      . A longitudinal study of internal medicine resident's retention of advanced cardiac life support skills. Acad Med 2006;81(Suppl 10):9–12.
      OpenUrlCrossRef

    Footnotes

    • Competing interests None.

    • Provenance and peer review Not commissioned; externally peer reviewed.