You are here

  • Home
  • Archive
  • Volume 10, Issue 5
  • Evaluation of previously embolized intracranial aneurysms: inter-and intra-rater reliability among neurosurgeons and interventional neuroradiologists

Article Text

Article menu
Download PDFPDF
Hemorrhagic stroke
Original research
Evaluation of previously embolized intracranial aneurysms: inter-and intra-rater reliability among neurosurgeons and interventional neuroradiologists
  1. Scott L Zuckerman1,
  2. Nikita Lakomkin2,
  3. Jordan A Magarik1,
  4. Jan Vargas3,
  5. Marcus Stephens4,
  6. Babatunde Akinpelu5,
  7. Alejandro M Spiotta3,
  8. Azam Ahmed6,
  9. Adam S Arthur7,
  10. David Fiorella8,
  11. Ricardo Hanel9,
  12. Joshua A Hirsch10,
  13. Ferdinand K Hui11,
  14. Robert F James12,
  15. David F Kallmes13,
  16. Philip M Meyers14,
  17. David B Niemann4,
  18. Peter Rasmussen15,
  19. Raymond D Turner3,
  20. Babu G Welch16,
  21. J Mocco2
  1. 1 Department of Neurosurgery, Vanderbilt University Medical Center, Nashville, Tennessee, USA
  2. 2 Department of Neurosurgery, Icahn School of Medicine at Mount Sinai, New York, USA
  3. 3 Department of Neurosurgery, Medical University of South Carolina, Charleston, South Carolina, USA
  4. 4 Department of Neurosurgery, University of Arkansas, Little Rock, Arkansas, USA
  5. 5 Department of Radiology, University of Washington, Washington, USA
  6. 6 Department of Neurosurgery, University of Wisconsin, Madison, Wisconsin, USA
  7. 7 Department of Neurosurgery, Semmes-Murphey Clinic, University of Tennessee Health Sciences Center, Memphis, Tennessee, USA
  8. 8 Department of Neurosurgery, Cerebrovascular Center, Stony Brook University Medical Center, Stony Brook, New York, USA
  9. 9 Department of Lyerly Neurosurgery, Baptist Neurological Institute, Jacksonville, Florida, USA
  10. 10 Neurointerventional Service, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts, USA
  11. 11 Department of Radiology, Johns Hopkins Hospital, Baltimore, Maryland, USA
  12. 12 Department of Neurosurgery, University of Louisville School of Medicine, Louisville, Kentucky, USA
  13. 13 Department of Neurosurgery, Mayo Clinic, Rochester, Minnesota, USA
  14. 14 Columbia University Medical Center, Departments of Neurosurgery and Radiology, New York, USA
  15. 15 Department of Neurosurgery, Cleveland Clinic, Cleveland, Ohio, USA
  16. 16 Department of Neurological Surgery, University of Texas Southwestern Medical Center, Dallas, Texas, USA
  1. Correspondence to J Mocco, Department of Neurological Surgery, Mount Sinai Health System, NY 10029, USA; j.mocco{at}mountsinai.org

Abstract

Background The angiographic evaluation of previously coiled aneurysms can be difficult yet remains critical for determining re-treatment.

Objective The main objective of this study was to determine the inter-rater reliability for both the Raymond Scale and per cent embolization among a group of neurointerventionalists evaluating previously embolized aneurysms.

Methods A panel of 15 neurointerventionalists examined 92 distinct cases of immediate post-coil embolization and 1 year post-embolization angiographs. Each case was presented four times throughout the study, along with alterations in demographics in order to evaluate intra-rater reliability. All respondents were asked to provide the per cent embolization (0–100%) and Raymond Scale grade (1-3) for each aneurysm. Inter-rater reliability was evaluated by computing weighted kappa values (for the Raymond Scale) and intraclass correlation coefficients (ICC) for per cent embolization.

Results 10 neurosurgeons and 5 interventional neuroradiologists evaluated 368 simulated cases. The agreement among all readers employing the Raymond Scale was fair (κ=0.35) while concordance in per cent embolization was good (ICC=0.64). Clinicians with fewer than 10 years of experience demonstrated a significantly greater level of agreement than the group with greater than 10 years (κ=0.39 and ICC=0.70 vs κ=0.28 and ICC=0.58). When the same aneurysm was presented multiple times, clinicians demonstrated excellent consistency when assessing per cent embolization (ICC=0.82), but moderate agreement when employing the Raymond classification (κ=0.58).

Conclusions Identifying the per cent embolization in previously coiled aneurysms resulted in good inter- and intra-rater agreement, regardless of years of experience. The strong agreement among providers employing per cent embolization may make it a valuable tool for embolization assessment in this patient population.

  • coil
  • aneurysm
  • angiography

https://doi.org/10.1136/neurintsurg-2017-013231

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    View Full Text

    Footnotes

    • Contributors SLZ designed the study, collected the data, wrote the statistical analysis plan, edited the manuscript, and is the guarantor. NL wrote the statistical analysis plan, cleaned the data, and drafted and edited the manuscript. JAM, JV, MS, BA, AMS, AA, ASA, DF, RH, JAH, FKH, RFJ, DFK, PMM, DBN, PR, RDT, and BGW collected the data and edited the manuscript. JM designed the study, collected the data, and edited the manuscript.

    • Funding This work was supported by Codman & Shurtleff, Inc.

    • Competing interests ASA is a consultant for Leica, Medtronic, Microvention, Penumbra, Siemens, and Stryker; has received research support from Microvention, Penumbra, and Siemens; and is a shareholder at Bendit, Cerebrotech, Serenity, and Synchron. BGW is a consultant for Stryker and Medtronic. PR is a member of the speaker’s bureau for Stryker Neurovascular; and a shareholder at Perflow Medical and Neurvana Medical. RDT has relationships with Codman, Medtronic, Penumbra, Microvention, Stryker, and Blockade Medical. JM is a consultant for Rebound Medical, Endostream, Synchron, and Cerebrotech; and an investor in Apama, The Stroke Project, Endostream, Synchron, Cerebrotech, NeurVana, and NeuroTechnology Investors.

    • Ethics approval Institutional review board approval was obtained for the current survey based cross sectional study.

    • Provenance and peer review Not commissioned; externally peer reviewed.