Background The adequacy of leptomeningeal collateral flow has a pivotal role in determining clinical outcome in acute ischemic stroke. The American Society of Interventional and Therapeutic Neuroradiology/Society of Interventional Radiology (ASITN/SIR) collateral score is among the most commonly used scales for measuring this flow. It is based on the extent and rate of retrograde collateral flow to the impaired territory on angiography.
Objective To evaluate inter- and intraobserver agreementin angiographic leptomeningeal collateral flow assessment.
Materials and methods Thirty pretreatment angiogram video loops (frontal and lateral view), chosen from the randomized controlled trial THRombectomie des Artères CErebrales (THRACE), were sent for grading in an electronic file. 19 readers participated, including eight who had access to a training set before the first grading. 13 readers made a double evaluation, 3 months apart.
Results Overall agreement among the 19 observers was poor (κ = 0,16 ± 6,5.10 -3), and not improved with prior training (κ = 0,14 ± 0,016). Grade 4 showed the poorest interobserver agreement (κ=0.18±0.002) while grades 0 and 1 were associated with the best results (κ=0.52±0.001 and κ=0.43±0.004, respectively). Interobserver agreement increased (κ = 0,27± 0,014) when a dichotomized score, ‘poor collaterals’ (score of 0, 1 or 2) versus ‘good collaterals’ (score of 3 or 4) was used. The intraobserver agreements varied between slight (κ=0.18±0.13) and substantial (κ=0.74±0.1), and were slightly improved with the dichotomized score (from κ=0.19±0.2 to κ=0.79±0.11).
Conclusion Inter- and intraobserver agreement of collateral circulation grading using the ASITN/SIR score was poor, raising concerns about comparisons among publications. A simplified dichotomized judgment may be a more reproducible assessment when images are rated by the same observer(s) in randomized trials.
Statistics from Altmetric.com
WBH and CM contributed equally.
Contributors WBH, SB, and ON planned the study data collection and identified the patient cohort. CM, WBH, and ON gathered the data; carried out the statistical analysis. All authors drafted and approved the final manuscriptand agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Funding This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient consent Not required.
Ethics approval CPP (Comité de Protection des Personnes) III Nord Est.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Data obtained from the THRACE study (NCT01062698). Any additional unpublished data are available upon request from the corresponding author