Article Text

Download PDFPDF
Original research
Inter- and intraobserver reliability for angiographic leptomeningeal collateral flow assessment by the American Society of Interventional and Therapeutic Neuroradiology/Society of Interventional Radiology (ASITN/SIR) scale
  1. Wagih Ben Hassen1,
  2. Claire Malley1,
  3. Grégoire Boulouis1,
  4. Frédéric Clarençon2,
  5. Bruno Bartolini3,
  6. Romain Bourcier4,
  7. Christine Rodriguez Régent5,
  8. Nicolas Bricout6,
  9. Marc Antoine Labeyrie7,
  10. Jean Christophe Gentric8,
  11. Aymeric Rouchaud9,
  12. Sébastien Soize10,
  13. Suzana Saleme11,
  14. Hélène Raoult12,
  15. Sophie Gallas13,
  16. François Eugène12,
  17. René Anxionnat14,
  18. Denis Herbreteau15,
  19. Serge Bracard16,
  20. Olivier Naggara17
  1. 1 Department of Neuroradiology, Centre Hospitalier Sainte Anne, Paris, France
  2. 2 Department of Interventional Neuroradiology, Hopital Universitaire Pitie Salpetriere, Paris, France
  3. 3 Hopitaux Universitaires Pitie Salpetriere-Charles Foix, Interventional Neuroradiology, Paris, France
  4. 4 Department of Diagnostic and Interventional Neuroradiology, Guillaume et René Laennec University Hospital, France
  5. 5 Department of Neuroradiology, Hospital Saint Anne, Paris, France
  6. 6 Department of Interventional Neuroradiology, Centre Hospitalier Regional Universitaire de Lille, Lille, France
  7. 7 Departement of Interventional Neuroradiology, Hopita Lariboisiere, Paris, France
  8. 8 CHRU de Brest, Brest, France
  9. 9 Centre Hospitalier Universitaire de Limoges, Limoges, France
  10. 10 Department of Radiology, University Hospital Reims, Reims, France
  11. 11 Department of Interventional Neuroradiology, CHU Limoges, Limoges, France
  12. 12 Department of Neuroradiology, University Hospital of Rennes, Rennes, France
  13. 13 Hopital Bicetre, Le Kremlin-Bicetre, France
  14. 14 Department of Neuroradiology, Centre Hospitalier Universitaire de Nancy, Nancy, France
  15. 15 Centre Hospitalier Regional Universitaire de Tours, Tours, France
  16. 16 Centre Hospitalier Universitaire de Nancy, Nancy, France
  17. 17 Department of Neuroradiology, Centre Hospitalier Sainte Anne, Paris, France
  1. Correspondence to Dr Olivier Naggara, Department of Neuroradiology, Centre Hospitalier Sainte, AnneParis 75014, France; O.NAGGARA{at}


Background The adequacy of leptomeningeal collateral flow has a pivotal role in determining clinical outcome in acute ischemic stroke. The American Society of Interventional and Therapeutic Neuroradiology/Society of Interventional Radiology (ASITN/SIR) collateral score is among the most commonly used scales for measuring this flow. It is based on the extent and rate of retrograde collateral flow to the impaired territory on angiography.

Objective To evaluate inter- and intraobserver agreementin angiographic leptomeningeal collateral flow assessment.

Materials and methods Thirty pretreatment angiogram video loops (frontal and lateral view), chosen from the randomized controlled trial THRombectomie des Artères CErebrales (THRACE), were sent for grading in an electronic file. 19 readers participated, including eight who had access to a training set before the first grading. 13 readers made a double evaluation, 3 months apart.

Results Overall agreement among the 19 observers was poor (κ = 0,16 ± 6,5.10 -3), and not improved with prior training (κ = 0,14 ± 0,016). Grade 4 showed the poorest interobserver agreement (κ=0.18±0.002) while grades 0 and 1 were associated with the best results (κ=0.52±0.001 and κ=0.43±0.004, respectively). Interobserver agreement increased (κ = 0,27± 0,014) when a dichotomized score, ‘poor collaterals’ (score of 0, 1 or 2) versus ‘good collaterals’ (score of 3 or 4) was used. The intraobserver agreements varied between slight (κ=0.18±0.13) and substantial (κ=0.74±0.1), and were slightly improved with the dichotomized score (from κ=0.19±0.2 to κ=0.79±0.11).

Conclusion Inter- and intraobserver agreement of collateral circulation grading using the ASITN/SIR score was poor, raising concerns about comparisons among publications. A simplified dichotomized judgment may be a more reproducible assessment when images are rated by the same observer(s) in randomized trials.

  • angiography
  • stroke
  • thrombectomy
  • standards

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.


  • WBH and CM contributed equally.

  • Contributors WBH, SB, and ON planned the study data collection and identified the patient cohort. CM, WBH, and ON gathered the data; carried out the statistical analysis. All authors drafted and approved the final manuscriptand agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

  • Funding This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None declared.

  • Patient consent Not required.

  • Ethics approval CPP (Comité de Protection des Personnes) III Nord Est.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data sharing statement Data obtained from the THRACE study (NCT01062698). Any additional unpublished data are available upon request from the corresponding author