You are here

Article Text

Article menu

Download PDFPDF

Neuroimaging
original article
Simulator based angiography education in neurosurgery: results of a pilot educational program
Free
  1. Kyle Michael Fargen1,
  2. Adnan H Siddiqui2,
  3. Erol Veznedaroglu3,
  4. Raymond D Turner4,
  5. Andrew J Ringer5,
  6. J Mocco1
  1. 1Department of Neurosurgery, University of Florida, Gainesville, Florida, USA
  2. 2Department of Neurological Surgery, University at Buffalo, Buffalo, New York, USA
  3. 3Department of Neurological Surgery, New Jersey Capital Health System, Trenton, New Jersey, USA
  4. 4Department of Neurological Surgery, Medical University of South Carolina, Charleston, South Carolina, USA
  5. 5Department of Neurosurgery, University of Cincinnati, Cincinnati, Ohio, USA
  1. Correspondence to Dr K M Fargen, Department of Neurosurgery, University of Florida, Box 100265, Gainesville, FL 32610, USA; kyle.fargen{at}neurosurgery.ufl.edu

Abstract

Introduction The use of simulators in medical training has been on the rise over the past decade as a means to teach procedural skills to trainees in a risk free environment. The goal of this study was to pilot a simulator based skills course for inexperienced neurosurgical residents to teach the fundamentals of cervicocerebral catheterization and angiography, with the ultimate goal of defining a universal simulator based curriculum that could be incorporated into neurosurgical resident training in the future.

Methods Seven neurosurgery residents with no prior angiographic experience served as the pilot participants for this 2 day course. Four neurointerventional trained neurosurgeons served as faculty for instruction and evaluation. The majority of the course focused on hands-on simulator practice with close mentoring by faculty. Participants were evaluated with pre-course and post-course assessments.

Results Post-course written test scores were significantly higher than pre-course scores (p<0.001). Faculty assessments of participants' technical skills with angiography (graded 0–10, with 10 being best) also improved significantly from pre-course to post-course (pre 2.1; post 5.9; p<0.001). Objective simulator recorded assessments demonstrated a significant decrease in the time needed to complete a four vessel angiogram (p<0.001) and total fluoroscopic time (p<0.001).

Conclusions Participant angiography skills, based on both faculty and simulator assessments, as well as participant knowledge, improved after this didactic, hands-on simulator course. Neuroendovascular simulator training appears to be a viable means of training inexperienced neurosurgery residents in the early learning stages of basic endovascular neurosurgery. Further studies evaluating the translation of procedural skills learned on the simulator to actual clinical skills in the angiography suite is necessary.

https://doi.org/10.1136/neurintsurg-2011-010128

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    The use of simulators in medical training has been on the rise over the past decade as a means of teaching procedural skills to trainees in a risk free environment.1 ,2 The increasing utility of simulation in residency training has been fueled both by rapidly advancing simulation technology and by the Accreditation Council of Graduate Medical Education (ACGME) requirements for resident proficiency based assessments.3 Although procedural simulators have been developed to teach the fundamentals of general surgery, vascular surgery and other surgical procedures, nowhere is the utility more realized than in the realms of endovascular and interventional radiology.3–5

    Unfortunately, the use of simulation in neurosurgical training appears to be lagging behind that of interventional radiology and general surgery. Resident training courses, such as those at the American Association of Neurological Surgeons and Congress of Neurological Surgeons meetings, use neurointerventional simulators but lack a curriculum based approach to instruction. The goal of this study was to pilot a simulator based skills course for inexperienced neurosurgical residents to teach the fundamentals of cervicocerebral catheterization and angiography, with the ultimate goal of defining a universal simulator based curriculum that could be incorporated into neurosurgical resident training in the future.

    Methods

    Course objectives and outline

    The primary objective was to provide resident participants with education in the fundamental principles of angiography. Secondary learning objectives were outlined such that participants were, at the termination of the course, able to: (1) identify neurovascular anatomy; (2) select appropriate catheters for diagnostic procedures; (3) identify different aortic arches; (4) navigate different aortic arches; (5) understand fundamentals of catheter and wire interaction; (6) discuss means to reduce radiation exposure to patients and physicians; and (7) perform a basic neurovascular angiogram on the simulator.

    The course was given over a 24 h period, with an initial evening lecture describing the course objectives followed by a lecture on fundamental neurovascular anatomy and radiation safety. The following day consisted of an instruction on basic simulator use, pre-course assessments, multiple hands-on simulator training and practice sessions, post-course assessments and finally an evaluation and feedback session.

    Participants

    Seven neurosurgery residents, with no prior angiographic and only minimal simulator experience (<1 h), from across the USA served as the pilot participants. Four neurointerventional trained neurosurgery assistant or associate professors, from four different institutions, served as faculty for instruction and evaluation. Each faculty member had previously completed an endovascular surgical neuroradiology fellowship and all are involved in training fellows at their home institution. Prior to the course, faculty attended an orientation meeting discussing the course goals, objectives and format, and were educated in participant assessments.

    Simulator

    Five Vist-C Simulator Systems (Mentice, Evanston, Illinois, USA) were used for this course allowing three participants sole access to their own simulator and four participants who worked as pairs on two simulators. The Vist-C Simulator System simulator utilizes actual catheters and wires that are engaged along internal tracking wheels and introduced through a port, allowing for the simulator to capture fine movements of the instruments both in the forward–backward plane as well as rotation. Fluoroscopy is simulated by a foot pedal and the bed position relative to the fluoroscopic device is controlled by a separate joystick.

    Education and course structure

    The course began with all participants taking a written test (detailed in the assessments section below), prior to starting an orientation dinner that included a basic lecture. This 1 h lecture covered fundamentals of angiography, some of which was not specifically tested in this pilot phase, such as groin access. However, the majority of this lecture detailed the specifics of brachiocephalic and intracranial vascular anatomy as well as descriptions of some basic techniques of wire and catheter manipulation. At the conclusion of this dinner, participants were given a 22 page paper supplement of basic angiography instruction developed specifically for this course and were instructed to prepare for the following day's instruction by reading the provided materials. The orientation lecture and supplementary reading addressed core principles, including: cerviocerebral anatomy, obtaining access, wire and catheter techniques and principles, prevention of air or thrombus embolism, radiation exposure and safety, contrast administration and basic fluoroscopic imaging views.

    The following morning began with a practical test on the simulator itself (detailed in the assessments section below), prior to beginning direct instruction. During the simulator based education portion of the course, a total of seven simulation practice cases were used, with increasingly difficult anatomic variables (two type I arches, one bovine arch and one type II arch, two type III arches). Each anatomic subset was introduced by a period of approximately 15 min of group instruction by a faculty member during which time that faculty member performed one of the two cases on the simulator while the images were projected onto a large screen for everyone to watch. While performing this case the faculty member detailed the specific techniques and critical steps required for safe performance of the angiogram. Additionally, the other faculty were present and would make additional comments and contributions as were felt to be appropriate and questions were encouraged from the participants. After each 15 min didactic session, participants went to their assigned simulator and began to practice the cases of that anatomic subset for 45 min. During this practice session the faculty spent all of their time moving among the participants, observing their technique and providing detailed instruction and guidance. After these first 3 h, the faculty left the participants without instruction for a 1 h period of instructor independent practice. A 1 h break was then given. Following this break, the faculty joined the participants for one more hour of personal instruction, after which time the participants were given 1 h of further independent practice. On completion of this hour, all participants were given a written test and a second practical test on the simulator itself (both detailed in the assessments section below). On completion of these tests, the participants were given anonymous surveys to evaluate the course and provide feedback. This completed all educational and assessment portions of the course.

    Assessments

    Participants were evaluated with three assessment tools. Firstly, participant knowledge was assessed with scored, 12 question, multiple choice written evaluations testing general principles of angiographic anatomy, procedures and indications. Questions asked participants to identify different types of catheters, aortic arches, angiographic anatomy and technical questions about proper fluoroscopy, catheter and wire advancement, and safety techniques. Identical written tests were given to each participant both before and at the termination of the course but no direct review of the test or its questions were performed during the course and participants were not told they would be receiving the same test at the end of the session. Identical tests were used so that the written test scores could be compared directly without bias from differing test difficulties.

    Secondly, participants were evaluated in a ‘practical test’ based on their performance during simulated diagnostic angiography on a single ‘patient’. Each participant was given 15 min maximum to perform a diagnostic angiogram on a test case patient with a type 1 aortic arch and moderate left internal carotid artery stenosis just distal to the common carotid artery bifurcation. At least two faculty members jointly evaluated each participant on a 10 point scale (0 being very poor and 10 being exceptional) based on their technical skills in catheter navigation, use of fluoroscopy and contrast, speed and other factors. A validated 10 point scale does not exist for measuring endovascular simulator skill and therefore a scoring system based on both qualitative and quantitative factors, as listed above, was devised and utilized. Additionally, the simulator itself recorded critical objective data, including time from catheter insertion to angiogram completion, amount of contrast used and amount of fluoroscopy time. Participants were assessed at two separate intervals: (1) during their initial simulated angiography case, which occurred after the introductory lecture and distribution of reading material (the night before) but prior to any formal one on one instruction; and (2) after completing the course and all one on one faculty–participant simulator practice sessions. Thirdly, participants were evaluated by simulator recorded objective criteria such as time to complete the study, amount of contrast used, total fluoroscopy time and potentially dangerous actions (catheter scraping against vessel wall, catheter advancing without supporting wire, wire in suboptimal vessel and guidewire passing lesion). Finally, participants submitted written evaluations of the course and were asked to provide feedback.

    Results

    Post-course written test scores were significantly higher than pre-course scores (number correct=10.6±0.5 vs 5.7±0.6, respectively; p<0.001), with a mean improvement of 4.9 correct questions. Faculty assessments of participants' technical skills with angiography also improved significantly from pre-course to post-course (pre 2.1±0.4; post 5.9±0.4; p<0.001). Objective simulator recorded assessments demonstrated a significant decrease in time needed to complete a four vessel angiogram (pre 15±0.9 min; post 9.5±1.1 min; p<0.001) and total fluoroscopic time (pre 10.8±1.0 min; post 6.3±1.2 min; p<0.001). Total contrast injected (in ml) was lower in the post-course angiogram although not significantly so (75.8±20.9 vs 85.0±15.0; p=0.46). Participant evaluations of the course were overwhelmingly positive (table 1).

    View this table:
    Table 1

    Results of participant completed course evaluations at the termination of the course

    Discussion

    The current exposure to neuroendovascular surgery among neurosurgical residency programs in the USA appears to be suboptimal. The ACGME 2009–2010 National Data Report for Neurological Surgery, a yearly document including averages of neurosurgical resident case logs across the country, reported a median of zero arteriography cases and a mean of 26.1 cases, indicating that most have little or no experience with angiography while a small minority have reported larger numbers.6 Unfortunately, the number of programs with associated endovascular neurosurgical fellowships is small, which in itself limits the experience of residents at most institutions. Currently, the ACGME website (http://www.acgme.org) lists only two ACGME accredited endovascular surgical neuroradiology programs, although many other programs do have fellowships that are not ACGME accredited. Furthermore, endovascular skills were identified as an area of inadequacy among practicing neurosurgeons applying for the oral board in a recent survey.7 Given the importance of endovascular diagnosis and treatment in the field of neurosurgery, it is imperative that resident exposure to such procedures is increased; however, doing so becomes increasingly challenging when one considers the continued expansion of neurosurgical procedures (complex spine, minimally invasive spine, deep brain stimulation, pain, peripheral nerve, endovascular) coupled with progressive reduction in the hours residents are allowed to work. Therefore, the onus is on neurosurgical educators to develop new and better ways to educate our residents to be able to perform more procedures despite less total time for learning. Simulation is likely to provide a crucial part of these new methods of education.8

    The use of simulation in endovascular training has risen sharply in the past decade due to exciting advances in simulator technology and improvements in simulator realism. New devices featuring ‘augmented reality’ simulation, which intertwines virtual reality with actual surgical devices, have been developed to provide improved haptic feedback and therefore more closely simulate actual clinical procedures.9–11 The correlation between clinical endovascular experience and simulator technical skills has been demonstrated in a variety of settings,12–19 suggesting that current simulators are able to mimic real conditions close enough that experience obtained in clinical practice translates to better simulator skills. Furthermore, there are data suggesting that the converse is true: that skills gained on simulators actually translate to improved clinical skills.17 Finally, there are data suggesting that even skilled interventional practitioners continue to benefit from simulation courses even after independent clinical practice.20 Therefore, a thorough understanding of the strengths and limitations of simulation devices is necessary such that the implementation of simulator based training into residency education can be optimized.21 ,22 The greatest limitations to simulator based education are the potential incongruence of simulator skills with actual clinical skills, high cost and poor tactile or haptic feedback. Although simulators may mimic real life procedures, actually demonstrating clinical benefit to simulation is difficult. In addition, the risk free simulation environment, in which poor technique or errors fail to generate real clinical consequences, may fail to encourage the caution needed in performing actual procedures. It is crucial, therefore, that appropriate feedback is provided to trainees such that maximal benefit can be obtained. In this course, we use both simulator and faculty feedback to enhance the learning experience of the participants.

    This study was conceived with the goal of designing a simulator based skills course for inexperienced neurosurgical residents to teach the fundamentals of cervicocerebral catheterization and angiography, providing the first steps toward the eventual goal of developing a universal simulator based curriculum that could be incorporated into neurosurgical resident training in the future. This is particularly important given both the increasing interest among neurosurgical residents in neuroendovascular surgery and the decreasing number of radiologists entering the field in the future.23 The 2 day resident course presented herein was successful, with all participants indicating that the course was beneficial in their training. Both written evaluations and faculty assessments of participant angiography skill improved significantly after instruction and simulator practice. Furthermore, simulator recorded assessments of total time to complete the procedure, fluoroscopy time and volume of contrast were improved at the termination of the course. Importantly, the objective simulator recorded data appropriately recapitulated expert faculty subjective grading, such that improvements in faculty generated scores were matched with improvements in simulator generated assessments. This relationship implies a low likelihood of confounding in faculty scoring secondary to examiner bias. However, it must be stated that a previous study using the simulator utilized in this course failed to correlate performances with experience level, with the exception of fluoroscopic time.24 Finally, participants were overwhelmingly positive about all aspects of the course in their course evaluations.

    In an era of expanded training requirements and vanishing duty hours, all available tools should be explored in an effort to improve our education methodology. Simulator based training may represent an important adjunct to training on real patients. Paramount to the acceptance of simulator training in residency training is the continued demonstration of skill transferability from the virtual device to real patients.25 Further research and evaluation of teaching courses like the one presented herein is essential in providing such data. Although these data are promising, expansion of such teaching courses to residency specific training curricula will be necessary before the full benefit of simulator training is realized. A further benefit of simulator based education efforts lies in the fact that demonstration of competency in various skills will likely become an ACGME requirement of neurosurgical residency programs in the near future. With continued development, simulator technology may represent a way to quantitate such competency evaluations.

    The small number of participants, lack of a control group and inability to correlate the simulator skills obtained through this course with skills on actual patients represent the major limitations to this study. Furthermore, the lack of a validated endovascular skills scoring scale introduces rater bias into faculty grading of participants.

    Conclusions

    Neuroendovascular simulator training appears to be a viable means of training inexperienced neurosurgery residents in the early learning stages of basic endovascular neurosurgery. This pilot simulator training course, focusing on cervicocerebral vascular anatomy, hands-on procedural simulator training with close faculty mentoring, and resident simulator practice, appears to be an effective method through which basic endovascular skills can be introduced. However, this is only a first step. Continued refinement of such pilot efforts are required, with the hope of bringing us closer to a valid curriculum that will have near universal application to neurosurgical training programs around the country.

    References

    1. ↵
      1. Ziv A,
      2. Wolpe PR,
      3. Small SD,
      4. et al
      . Simulation-based medical education: an ethical imperative. Acad Med 2003;78:783–8.
      OpenUrlCrossRefPubMedWeb of Science
    2. ↵
      1. Issenberg SB,
      2. McGaghie WC,
      3. Petrusa ER,
      4. et al
      . Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach 2005;27:10–28.
      OpenUrlCrossRefPubMedWeb of Science
    3. ↵
      1. Desser TS
      . Simulation-based training: the next revolution in radiology education? J Am Coll Radiol 2007;4:816–24.
      OpenUrlCrossRefPubMed
    4. ↵
      1. Gould DA
      . Interventional radiology simulation: prepare for a virtual revolution in training. J Vasc Interv Radiol 2007;18:483–90.
      OpenUrlCrossRefPubMedWeb of Science
    5. ↵
      1. Tsang JS,
      2. Naughton PA,
      3. Leong S,
      4. et al
      . Virtual reality simulation in endovascular surgical training. Surgeon 2008;6:214–20.
      OpenUrlCrossRefPubMedWeb of Science
    6. ↵
      1. Bose A,
      2. Henkes H,
      3. Alfke K,
      4. et al
      . The Penumbra System: a mechanical device for the treatment of acute stroke due to thromboembolism. AJNR Am J Neuroradiol 2008;29:1409–13.
      OpenUrlAbstract/FREE Full Text
    7. ↵
      1. Mazzola CA,
      2. Lobel DA,
      3. Krishnamurthy S,
      4. et al
      . Efficacy of neurosurgery resident education in the new millennium: the 2008 Council of State Neurosurgical Societies post-residency survey results. Neurosurgery 2010;67:225–32.
      OpenUrlCrossRefPubMedWeb of Science
    8. ↵
      1. Bath J,
      2. Lawrence P
      . Why we need open simulation to train surgeons in an era of work-hour restrictions. Vascular 2011;19:175–7.
      OpenUrlCrossRefPubMedWeb of Science
    9. ↵
      1. Lemole GM Jr,
      2. Banerjee PP,
      3. Luciano C,
      4. et al
      . Virtual reality in neurosurgical education: part-task ventriculostomy simulation with dynamic visual and haptic feedback. Neurosurgery 2007;61:142–8.
      OpenUrlPubMed
    10. ↵
      1. Botden SM,
      2. Jakimowicz JJ
      . What is going on in augmented reality simulation in laparoscopic surgery? Surg Endosc 2009;23:1693–700.
      OpenUrlCrossRefPubMed
    11. ↵
      1. Botden SM,
      2. Torab F,
      3. Buzink SN,
      4. et al
      . The importance of haptic feedback in laparoscopic suturing training and the additive value of virtual reality simulation. Surg Endosc 2008;22:1214–22.
      OpenUrlCrossRefPubMed
    12. ↵
      1. Bech B,
      2. Lonn L,
      3. Falkenberg M,
      4. et al
      . Construct validity and reliability of structured assessment of endovascular expertise in a simulated setting. Eur J Vasc Endovasc Surg 2011;42:539–48.
      OpenUrlCrossRefPubMed
    13. ↵
      1. Van Herzeele I,
      2. Aggarwal R,
      3. Choong A,
      4. et al
      . Virtual reality simulation objectively differentiates level of carotid stent experience in experienced interventionalists. J Vasc Surg 2007;46:855–63.
      OpenUrlCrossRefPubMedWeb of Science
    14. ↵
      1. Van Herzeele I,
      2. Aggarwal R,
      3. Malik I,
      4. et al
      . Validation of video-based skill assessment in carotid artery stenting. Eur J Vasc Endovasc Surg 2009;38:1–9.
      OpenUrlPubMed
    15. ↵
      1. Dayal R,
      2. Faries PL,
      3. Lin SC,
      4. et al
      . Computer simulation as a component of catheter-based training. J Vasc Surg 2004;40:1112–17.
      OpenUrlCrossRefPubMed
    16. ↵
      1. Tedesco MM,
      2. Pak JJ,
      3. Harris EJ Jr,
      4. et al
      . Simulation-based endovascular skills assessment: the future of credentialing? J Vasc Surg 2008;47:1008–14.
      OpenUrlCrossRefPubMed
    17. ↵
      1. Chaer RA,
      2. Derubertis BG,
      3. Lin SC,
      4. et al
      . Simulation improves resident performance in catheter-based intervention: results of a randomized, controlled study. Ann Surg 2006;244:343–52.
      OpenUrlPubMedWeb of Science
    18. ↵
      1. Dawson DL
      . Training in carotid artery stenting: do carotid simulation systems really help? Vascular 2006;14:256–63.
      OpenUrlAbstract/FREE Full Text
    19. ↵
      1. Botden SM,
      2. de Hingh IH,
      3. Jakimowicz JJ
      . Meaningful assessment method for laparoscopic suturing training in augmented reality. Surg Endosc 2009;23:2221–8.
      OpenUrlPubMed
    20. ↵
      1. Van Herzeele I,
      2. Aggarwal R,
      3. Neequaye S,
      4. et al
      . Experienced endovascular interventionalists objectively improve their skills by attending carotid artery stent training courses. Eur J Vasc Endovasc Surg 2008;35:541–50.
      OpenUrlCrossRefPubMedWeb of Science
    21. ↵
      1. Gould D
      . Using simulation for interventional radiology training. Br J Radiol 2010;83:546–53.
      OpenUrlAbstract/FREE Full Text
    22. ↵
      1. Gould DA,
      2. Kessel DO,
      3. Healey AE,
      4. et al
      . Simulators in catheter-based interventional radiology: training or computer games? Clin Radiol 2006;61:556–61.
      OpenUrlCrossRefPubMed
    23. ↵
      1. Strozyk D,
      2. Hanft SJ,
      3. Kellner CP,
      4. et al
      . Training in endovascular surgical neuroradiology. World Neurosurg 2010;74:28–31.
      OpenUrlPubMed
    24. ↵
      1. Berry M,
      2. Lystig T,
      3. Reznick R,
      4. et al
      . Assessment of a virtual interventional simulator trainer. J Endovasc Ther 2006;13:237–43.
      OpenUrlCrossRefPubMed
    25. ↵
      1. Ahmed K,
      2. Keeling AN,
      3. Fakhry M,
      4. et al
      . Role of virtual reality simulation in teaching and assessing technical skills in endovascular intervention. J Vasc Interv Radiol 2010;21:55–66.
      OpenUrlCrossRefPubMedWeb of Science

    Footnotes

    • Competing interests None.

    • Ethics approval Ethics approval was provided by UF IRB.

    • Provenance and peer review Not commissioned; externally peer reviewed.