Object Despite the popularity of flow-diverting stents for the treatment of cerebral aneurysms, there is no widely accepted scale for the characterization of results. We present an outcomes-based grading scale that considers factors related to failure of flow diversion.
Methods The grading scale was developed using the results from consecutive patients at two institutions who were treated with flow diversion for a cerebral aneurysm. The initial treatment results were graded on patient, aneurysm, and treatment characteristics. A 6-point grading scale was developed based on these data.
Results One hundred and seventy-one patients were included in the patient cohort. When compared by multivariate analysis with patients without residuals, patients with aneurysm residuals were found to be older (age ≥60 years, p=0.01, OR 1.17, 95% CI 1.03 to 1.33), to have larger aneurysms (size ≥15 mm, p<0.01, OR 1.38, 95% CI 1.17 to 1.62), to have aneurysms with associated side branches (p=0.02, OR 1.17, 95% CI 1.03 to 1.33), and to have a post-treatment Raymond score of 2 or 3 (p=0.01, OR 1.28, 95% CI 1.06 to 1.56). Using the Raymond score (1–3) as the foundation for the grading scale, additional points (0 or 1) were given for the other three identified factors, creating a 6-point scale. We found that patients with residual aneurysms had statistically higher final tabulated scores (p<0.01).
Conclusions We propose a novel straightforward outcomes-based scale to characterize results after flow diversion treatment of cerebral aneurysms. This scale may provide the basis for the common reporting of results in future studies.
- Flow Diverter
Statistics from Altmetric.com
The advent of flow diversion has revolutionized the endovascular treatment of cerebral aneurysms. Although the initial approval for the Pipeline Embolization Device (Covidien/Medtronic) was predicated on its use in large or giant cavernous or paraclinoid internal carotid artery aneurysms, the use of flow-diverting stents has rapidly increased to encompass cerebral aneurysms of smaller size and in other vessel locations.1–6 Furthermore, additional devices have since been brought to the market, with several more undergoing testing for regulatory approval.7–9 As this new technology gains further traction, there is a strong need for a common language with which to describe treatment results and standardize reporting of clinical results and trials.
Currently, there are three published grading systems for the classification of aneurysms after flow diversion treatment.10–12 The OKM, Kamran-Byrne, and SMART scales were developed based solely on the initial treatment results of the aneurysm without consideration of final outcomes. As such, their usefulness in predicting final angiographic outcomes has not been tested. To date, these systems have not been widely adopted in the medical literature and their inter-rater reliability is low.13
We present a novel straightforward outcomes-based grading scale to classify aneurysms treated with flow diversion. The Flow Diversion Stent Score (FDSS) incorporates several easy-to-categorize variables that have been hypothesized to influence outcomes after flow diversion.
We queried retrospective databases at each participating institution (University of Utah Hospital, Salt Lake City, Utah, and Barrow Neurological Institute, Phoenix, Arizona, USA) to identify consecutive patients treated with flow-diverting stents from August 2012 to September 2015. We classified these patients into two cohorts: those with aneurysms with residuals at a minimum of 12 months of follow-up and those with aneurysms without residuals regardless of length of follow-up. Aneurysm residual was defined as contrast opacification of the aneurysm neck or dome on follow-up CT angiography, magnetic resonance angiography, or catheter angiography. Patients without any follow-up were excluded from the analysis, as were patients with aneurysm residuals without the requisite length of follow-up. Additionally, any patient who underwent retreatment of the target aneurysm for residuals regardless of length of follow-up was included in the residual cohort.
Patients were analyzed based on standard demographic variables (ie, age, sex), aneurysm characteristics (ie, anterior vs posterior circulation, size, presence of a side branch), and treatment results (ie, treatment strategy including coil embolization and/or multiple devices, Raymond score, presence of contrast stasis within the aneurysm, presence of residual aneurysm at last follow-up).
Cohort characteristics and surgical details were summarized overall and by outcome depending on whether residual aneurysm was present after flow diversion. χ2 tests and t-tests were used for univariate analysis on categorical and continuous variables, respectively, to identify potential predictors for residual aneurysm after flow diversion. Variables that met statistical significance (p<0.05) or were near significance (p<0.1) on univariate analysis were then included in a multivariate logistic regression model. Variables that met statistical significance on the multivariate model were considered in the creation of the scoring system. The internal validity of the scoring system was assessed using a bootstrapping resampling methodology. Inter-rater reliability was assessed by calculating Kendall's correlation of concordance (W) with correction for ties on a subset of the study sample.14 This provided a test to determine whether there was concordance among three independent raters regarding which patients had the highest FDSS scores. Statistical analysis was performed using R software (V.3.2.0, R Foundation for Statistical Computing; http://www.R-project.org).
A total of 171 patients met the inclusion criteria for our analysis (table 1). There were 46 (26.9%) patients in the cohort with residual aneurysms, with an average age of 63.6±14.3 years (range 16–83 years), whereas there were 125 (73.1%) patients in the cohort without residual aneurysms, with an average age of 57.6±13.9 years (range 23–82 years). In the residual cohort, 42 of 46 (91.3%) aneurysms were located in the anterior circulation, whereas 107 of 125 (85.6%) aneurysms were located in the anterior circulation in the no residual cohort. The average size of aneurysms with residuals was 13.0±9.3 mm and that of aneurysms without residuals was 9.3±6.2 mm. Mean±SD length of follow-up for aneurysms with residuals was 15.0±11.9 months.
On univariate analysis, age ≥60 years (p=0.02), aneurysm size ≥15 mm (p<0.01), and Raymond score 2 or 3 (p<0.01) had statistically significant associations with residual aneurysm on follow-up angiography (table 2). We examined both 10 mm and 15 mm as size cut-offs for our grading scale; however, only the 15 mm size cut-off was found to be statistically significant on univariate analysis. The presence of a side branch (p=0.09) and lack of coil use (p=0.06) were near statistical significance. These variables were all included in the multivariate analysis. Sex (p=0.94), aneurysm location in the anterior circulation (p=0.47), lack of contrast stasis (p=0.33), and use of a single flow-diverting stent (p=0.45) were not associated with residual aneurysms.
On multivariate analysis, aneurysm size ≥15 mm (OR 1.38, 95% CI 1.17 to 1.62), Raymond score 2 or 3 (OR 1.28, 95% CI 1.06 to 1.56), age ≥60 years (OR 1.17, 95% CI 1.03 to 1.33), presence of side branch (OR 1.17, 95% CI 1.03 to 1.33), and lack of coil use (OR 1.17, 95% CI 1.02 to 1.34) were independent risk factors for residual aneurysms on follow-up angiography (table 3).
Creation of FDSS grading system
After careful analysis of the results we developed an outcomes-based grading system for aneurysms treated with flow diversion. From among the factors identified as significant on multivariate analysis, we selected easily identifiable variables to minimize any confusion with the system. The foundation of the grading system is the initial Raymond score (1–3) after the treatment. We included three additional elements which we identified as statistically significant factors related to aneurysm occlusion after flow diversion: aneurysm size, presence of side branches, and patient age.
Grading system components: the Raymond score
Our grading scale uses the 3-point Raymond score as the foundation, because of its widespread acceptance.15 It has the added benefit of identifying aneurysms that have complete or near-complete occlusion at the time of treatment. Generally, most aneurysms will demonstrate some element of dome filling after the initial treatment with flow diversion. The use of the Raymond score also can account for differences in treatment strategy. Use of multiple overlapping devices and/or adjunctive coil embolization can have a significant effect on the degree of aneurysm filling after treatment.16 This also eliminates the need to include treatment-specific variables (ie, adjunctive coil embolization and/or multiple devices) to the scoring system.
Grading system components: side branches
The presence of side branches has also been posited as a mechanism for failure of aneurysm occlusion after flow diversion.17 While flow-diverting stents were designed to maintain patency of covered side branches,2 ,4 ,18–20 this crucial aspect of the device may adversely affect the end result of aneurysm occlusion. Continued demand through a side branch associated with the aneurysm—either at the neck, sidewall, or dome—may result in incomplete occlusion of the aneurysm because of the preservation of flow within the aneurysm sac. Indeed, we identified the presence of a side branch—namely, any angiographically visible branching artery associated with the neck, sidewall, or dome of the aneurysm—as a statistically significant difference between those aneurysms that went on to complete occlusion and those that did not. The presence of side branches was included as a binary variable (yes=1, no=0).
Grading system components: aneurysm size
The size of the aneurysm is also likely related to the final outcomes after flow diversion.1 ,21 ,22 Based on our analysis of the series, the aneurysm size cut-off was set at 15 mm, with aneurysms <15 mm rating as 0 and aneurysms ≥15 mm rating as 1.
Grading system components: patient age
Interestingly, the age of the patient was identified as a statistically significant variable in our analysis. As with aneurysm size, patient age was included as a binary variable (patient age ≥60 years=1, patient age <60 years=0). The demarcation point of 60 years was selected based on the mean ages of the two cohorts. Our analysis is the first to identify patient age as a determinant of outcomes after flow diversion. This unique finding may be related to the overall health of older patients and a potentially reduced ability to develop adequate endothelialization of the device; however, at the time of writing, this hypothesis is purely conjectural.
The final score is determined by adding the values from each of the four variables. The lowest score (FDSS 1) would represent a small aneurysm without side branches in a younger patient that was completely occluded after placement of the flow-diverting stent. The highest score (FDSS 6) would represent a large aneurysm with side branches in an older patient that completely fills at the time of initial treatment. Figure 1 and online supplementary figures 1–3 demonstrate application of the FDSS.
Test performance and validation
We applied this grading system to our series of aneurysms to determine whether the scale accurately predicted patient outcomes. Our two cohorts demonstrated a statistically significant difference with regard to the final score between the two cohorts: patients with aneurysms with a residual had an average score of FDSS 4.4±0.7 whereas those with completely occluded aneurysms had an average score of FDSS 3.6±1.0 (p<0.01) (see online supplementary table 1). Of the 46 residual aneurysms, 45 had a score of ≥4 (sensitivity 97.8%). Despite these results, a fair number of aneurysms that were graded FDSS ≥4 went on to have complete occlusion (66 of 116 (56.9%); see online supplementary table 2). Thus, using a cut-off score of 4 resulted in a specificity of 50.9% and a positive predictive value of 40.5%—that is, patients with a score of ≥4 had a 40.5% chance of having an aneurysm residual on follow-up compared with a 26.9% chance for the entire study population. The inter-rater reliability of the FDSS (Kendall's W) was 0.61 (p=0.005).
We tested the internal validity of the residual aneurysm scoring system by using a bootstrapping technique. Samples were from the residual aneurysm cohort. For each sample, the mean FDSS for residual aneurysm was calculated. Based on 1000 bootstrap replicates, the mean score was FDSS 4.44±0.10 (95% CI 4.24 to 4.64). Thus, our data support using a score of 4 for predicting which aneurysms are more likely to result in residual filling on follow-up angiography.
Based on assessment of risk factors for residual aneurysms, we calculated the FDSS according to age ≥60 (1 point), aneurysm size ≥15 mm (1 point), presence of side branch (1 point), and post-treatment Raymond score (1–3 points). Patients with FDSS ≥4 were more likely to have residual aneurysm filling on follow-up angiography.
Currently, there are three published grading systems (OKM, Kamran-Byrne, and SMART scales) for the characterization of aneurysms treated with flow diversion.10–12 The Kamran-Byrne and SMART scales share common authors. To date, none of these three scales has been widely adopted in the medical literature. The authors developed their respective systems based on observations of initial treatment results. Although they selected variables based on likely factors related to final treatment results (eg, presence of contrast stasis, degree of initial aneurysm occlusion), they did not specifically examine their systems based on the final treatment results. Unlike these competing scales, we selected the variables for inclusion based on both the initial and the final treatment results.
Additionally, we sought to increase inter- and intra-rater reliability by selecting readily identifiable variables. We have previously published the inter-rater reliability of the three published scales using our consecutive series of flow-diverted aneurysms.13 In our analyses there was generally poor or fair reliability when using any of the three grading systems. We hypothesized that this was likely related to the complexity of the published scales. There was particular difficulty in classification of the degree of contrast stasis using these scales.
Although we included contrast stasis in our initial analysis, we did not identify a statistically significant difference as it relates to final outcomes. For our purposes, we defined contrast stasis as the presence of contrast in the aneurysm after contrast clears the parent artery. This simplified definition of contrast stasis was selected in an effort to increase the inter-rater reliability.
While we identified the use of adjunctive coil embolization as a statistically significant variable in multivariate analysis, we elected to exclude it from our grading scale for several reasons. First, while coiling with flow diversion is a widely published practice, it still represents only a minority of cases, with <15% of patients treated with coiling in the International Retrospective Study of Pipeline Embolization Device (IntrePED).23 Additionally, since it seems more likely that aneurysms treated with coil embolization will have better initial occlusion results, we felt that the Raymond score would be able to capture a portion of these patients. Finally, as we mentioned previously, we wanted to minimize the complexity of any grading scale.
A number of hypotheses have been proposed to account for the failure of aneurysm occlusion after flow diversion. The presence of a previous stent has been identified as a possible factor limiting occlusion of an aneurysm after flow diversion.24–26 Additionally, the presence of associated side branches has been postulated to limit complete aneurysm occlusion.17 Continued inflow due to a side branch may affect the device's ability to decrease the inflow into the aneurysm and also limit the degree of stasis in the aneurysm, thus negatively affecting the final treatment results. This may be more pronounced when the side branch is situated at the side wall or dome of the aneurysm than at the neck. Poor wall apposition of the device can also limit its ability to adequately divert flow away from the aneurysm and, thus, limit the efficacy of treatment. Finally, the role of antiplatelet regimens on the long-term results of aneurysm occlusion has not been studied in any detail and should be evaluated.
In theory, any factors that may account for potential failure should be considered in a grading system, but although some of these factors might contribute to failure of flow diversion, we chose to limit the complexity of our proposed grading system. Therefore, we did not examine the presence of previous stents, the location (neck, side wall, or dome) of associated side branches to the aneurysm, inadequate wall apposition, or antiplatelet regimens as factors in our analysis. Additionally, certain variables may expose a more subjective element to the scoring system with the potential to adversely affect the inter-rater reliability of the scale. We sought to minimize any possible confusion by selecting variables that lend themselves to more uniform assessment among raters.
The inter-rater reliability of the FDSS as assessed by three independent graders was acceptable (W=0.61). The data included all aneurysms treated with flow diversion from two institutions over the study period, which increases the generalizability of the results. However, external validation of this grading scale using data from other institutions is needed. The decision to proceed with flow diversion was up to the treating surgeon, which may introduce selection bias. For example, some surgeons may prefer flow diversion for certain aneurysm types and morphology, and these biases may affect our study results. There is also variability in the timing of follow-up and the imaging modality used.
Few medical grading systems can claim to have near-perfect sensitivity and specificity, and ours is a far from perfect characterization of flow-diverted aneurysms. It is possible that the addition of more variables could improve the predictive value of our scale. However, the inclusion of more variables results in a more elaborate grading system. The desire to make the most accurate system must be weighed against the ease of use to maximize the possibility of widespread adoption. We have attempted to create such a system by using an outcomes-based approach rather than the descriptive ones previously published. Our proposed scale may provide a common language that can be used in future reporting of results. This scale will require independent validation prior to any adoption.
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
MSP and MDM are joint first authors.
Contributors Concept and design: MSP, CGMcD. Data collection: MSP, MDM, KM, MJN, LMS, BW. Manuscript preparation: MSP, MDM. Editorial oversight: MSP, FCA, PT, JRK, CGMcD. Final approval: MSP, CGMcD.
Competing interests PT is a consultant for Covidien and proctors physicians in the use of the PED.
Ethics approval Ethics approval was obtained from the University of Utah and Barrow Neurological Institute institution review boards.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.