Article Text
PDF
Abstract
Background and purpose The Raymond–Roy grading scale is used for aneurysm coiling with only limited data on its validity. The scale was developed based on the extent of initial aneurysm occlusion from 1 to 3. However, the model usefulness in evaluating recurrence, retreatment, and rebleeding is unknown. Our goal was to perform a meta-analysis to evaluate the predictiveness of the Raymond scale.
Methods We performed a systematic review of the English literature for aneurysm coiling which reported the initial embolization results, based on the Raymond–Roy grading scale, and the respective recurrence rates, retreatment rates, and rebleed rates. This yielded data for 4587 aneurysms. We conducted a Bayesian random effects meta-analysis to evaluate the outcomes with respect to the reported initial embolization results.
Results We found the Raymond scale to be predictive of retreatment, with statistically higher rates of retreatment with higher initial Raymond grade. Furthermore, we found a higher probability of rebleeding for initial grades 2 or 3 versus grade 1, which approached significance. The rebleed rates were probably affected by monitoring and treatment of recurrence. However, although there was a trend towards higher recurrence rates with initial grade, this was not statistically significant.
Conclusions The modified Raymond–Roy scale appears to provide reasonable predictive value for treated aneurysm, especially for the clinically more important aspects of retreatment and rebleed rates.
- Aneurysm
- Coil
- Intervention
- Statistics
- Subarachnoid
Statistics from Altmetric.com
Introduction
Given the large and growing number of endovascularly treated aneurysms, there is a need for a clinically predictive aneurysm occlusion grading system. The modified Raymond–Roy scale is one of the most commonly used methods for grading aneurysm occlusion following embolization. This classification defines the completeness of endosaccular occlusion on a three point scale—Grade 1: complete occlusion; Grade 2: neck remnant; and Grade 3: residual aneurysm flow.1 Although the inter-observer objectivity may be improved compared with other grading systems with more numerous divisions, its prognostic significance in terms of predictive clinical outcome and long-term durability has not been as well established.2 ,3 To date, no study or meta-analysis has sufficiently evaluated the clinical predictability of this grading system. There is a tendency of higher recurrence after aneurysm coiling with higher grades on initial aneurysm coiling; however, the retreatment rates and rebleed rates are less well understood. We conducted a Bayesian random effects meta-analysis to study the recurrence, retreatment, and rebleed rates of aneurysms treated with coil embolization with respect to the reported initial aneurysm embolization results as graded by the modified Raymond–Roy score. Our aim was to determine if there is a statistically significant difference in the evolution of these aneurysms depending on the grade of initial occlusion.
Methods
We performed a PubMed database search using variations of the terms aneurysm, embolization, coiling, recurrence, and retreatment. This yielded 1312 articles. We further screened for reports that provided data on initial aneurysm coiling as well as follow-up CT, MR, or conventional angiogram results using the Raymond–Roy system or some closely defined variation of that scale. We also wanted to select studies with at least 1 year follow-up. Given staged treatment approaches, we defined recurrence or retreatment as those occurring after more than 3 months to exclude intentionally incomplete treatment. Moreover, recurrence was defined within our cohort of studies as recanalization which upgraded the Raymond–Roy grade or enlargement within a grade resulting in retreatment. Retreatment may have been either endovascular coiling or surgical clipping. Rebleed was defined as intracranial hemorrhage occurring more than 1 month after embolization in order to exclude periprocedural complications. Case series with fewer than 10 subjects and case reports were excluded from analysis.
In analyzing the published literature we found variability in the aneurysm occlusion grading system and included studies where the description could be translated into the Raymond–Roy grade. For example, a description of ‘neck remnant’ was categorized as Raymond–Roy grade 2 even though the article did not specifically refer to the Raymond–Roy grading system. Conversely, articles which reported rates without the above described three tier grading, or those that reported residual or recurrent aneurysm lumen as volumetric percentages of the total aneurysm, were excluded.
Additionally, many of the articles had not presented the recurrence, retreatment, or rebleed rates with respect to the initial results in table format and therefore these data were extrapolated from within the body of their text. Articles with overall insufficient data were excluded. However, in one case the retreatment and rebleed rates were reported but not the recurrence rates and we made the assumption that retreatment rates reflected at least this number of recurrences. Articles which provided complete data although may not have explicitly stated any rebleeds were assumed to indicate no rebleeds.
We fit a (Bayesian) random effects model with diffuse prior distributions separately to the recurrence rate, retreatment rate, and rebleed rate. Random study effects were incorporated, as well as a random effect for each initial Raymond–Roy grade, within each study. Because the large majority of studies included a single aneurysm per patient, we treated aneurysms as independent. Within these models we compared the posterior probabilities of recurrence, retreatment, and rebleeding, given the initial procedural Raymond–Roy score.
Bayesian random effects model for meta-analysis
The model for each response (recurrence, retreatment, or rebleeding) is given below:
In this model yij is the number of recurrences, retreatments, or rebleeds in study i and with initial grade j. yij is assumed to be distributed binomially with index nij and probability pij. The logit (log odds) of pij is a linear function with a random term that represents the initial grade j for study i (βij) and a random effect for study (bi). The following Gaussian priors are used for the study effects and initial grade within study effects, along with their priors and corresponding diffuse hyperpriors:
For interpretation, the effect of initial grade j within study i is distributed normally with mean as the average effect of initial grade j βj over all the studies. βj is distributed normally with mean β0 and variance 
. The between-study precision, 
, and between-grade precision, 
, are distributed gamma with shape and scale parameters equivalent to a prior representing little prior knowledge.
We modeled the effect of initial grade as being potentially different within studies because, when we assumed the effect was constant across studies, there was too much shrinkage of some of the posterior probabilities toward the average, especially for the study/initial grade combinations that experienced zero observed events (the Results section provides more information about shrinkage from random effects models). In addition, typical measures of goodness of fit (eg, Deviance Information Criterion (DIC)) favored the model we used over one with a constant effect of initial grade. However, we make inferences on the βj values because they are the population averages. For example, we report the study-averaged probabilities for initial grade j, exp(βj)/(1+exp(βj)), below.
Results
Ten studies involving 4587 treated cerebral artery aneurysms provided data with respect to the selection criteria.4–13 The mean follow-up for patients reported in these studies was 2.6 years, ranging from 6 months to 11 years. The pooled initial angiographic results after aneurysm coiling using the Raymond–Roy scale as well as the recurrence, retreatment, and rebleed rates are shown in table 1. The combined initial results were similar to the individually reported rates: grade 1, 62%; grade 2, 29%; grade 3, 8%. Furthermore, the overall pooled recurrence and retreatment rates for each grade are also similar to the individually reported rates.
Combined reported initial embolization results, recurrences, retreatments, and rebleeds from 10 included studies
For those aneurysms with initial grade 1 occlusion there was 14% recurrence, 6% retreatment, and <1% rebleeding, for aneurysms with initial grade 2 neck remnant there was 20% recurrence, 14% retreatment, and <1% rebleeding, and for aneurysms with initial grade 3 residual aneurysm there was 20% recurrence, 24% retreatment, and 1% rebleeding.
We computed the posterior probabilities of recurrence, retreatment, and rebleeding for each initial grade, as well as the probability that the higher grade has a higher recurrence, for each pair of grades. Table 2 shows that the posterior mean of the study-averaged probability of recurrence is 0.15 for initial grade 1, 0.18 for initial grade 2, and 0.16 for initial grade 3, with 95% credible intervals in parentheses. The posterior probability that the study-averaged probability of recurrence is greater for initial grades 2 and 3 compared with grade 1 is 0.73 and 0.58, respectively. The posterior probability that the study-averaged probability of recurrence is greater for initial grade 3 compared with grade 2 is only 0.35.
Posterior means and credible intervals of study-averaged probabilities of recurrence, retreatment, and rebleed, as well as comparative statistical inference
In the third row of table 2 we show the posterior probability that an initial grade 1 is less likely to have recurrence than either grade 2 or grade 3. Because this posterior probability is only 0.82, it may be concluded that recurrence is not significantly more likely for grades 2 or 3 compared with grade 1, if we consider a 0.95 posterior probability as implying statistical significance.
Figure 1 plots the posterior means and 95% credible intervals of the study-specific probabilities of recurrence by study and initial aneurysm grade (red), along with the observed proportions and the associated 95% CIs (black). Horizontal lines represent the posterior means of the study-averaged probabilities of recurrence for grade 1 (solid), grade 2 (long dash), and grade 3 (short dash). One feature of random effects models is that of shrinkage of the estimated probabilities toward one another, down-weighting the effects of outliers. This shrinkage is seen in figure 1 where the red dashes for zero rates are less extreme (closer to the average) and the red lines extend generally narrower than the corresponding black lines. Figure 2 shows this shrinkage in a different way for the estimated recurrence probabilities by study and initial aneurysm grade. The left panel of figure 2 shows the observed proportions of recurrence and the right panel shows the posterior mean probabilities. Note that observed proportions that were zero (perhaps due to not enough samples) are no longer estimated to have a zero rate when the Bayesian random effects model is fit.
Estimated probability of recurrence, with 95% uncertainty intervals, by study and grade.
Shrinkage of (study, initial grade) proportions after model fit.
A similar situation holds for retreatment and rebleeding. Table 2 shows that the posterior mean of the study-averaged posterior probability of retreatment is 0.05 for initial grade 1, 0.11 for initial grade 2, and 0.23 for initial grade 3, along with 95% credible intervals. The posterior probability that the study-averaged probability of retreatment is greater for initial grades 2 and 3 compared with grade 1 is 0.97 and 0.99, respectively. The posterior probability that the study-averaged probability of retreatment is greater for initial grade 3 than for grade 2 is 0.98. As with recurrence, we show the study-averaged posterior probability that an initial grade 1 is less likely to have retreatment than either grade 2 or grade 3. As shown, grade 1 is significantly less likely to have a retreatment than either grades 2 or 3 (probability of 0.99). Furthermore, the order of retreatment occurrence is highly like to be monotone with respect to grade (probability of 0.945).
Figure 3 plots the posterior means and 95% credible intervals for the probabilities of retreatment by study and initial aneurysm grade (red), along with the observed proportions and the associated 95% CIs (black). Again, horizontal lines represent the posterior means of the study-averaged probabilities of recurrence for grade 1 (solid), grade 2 (long dash), and grade 3 (short dash).
Estimated probability of retreatment, with 95% uncertainty intervals, by study and grade.
The posterior mean of the study-averaged probability of rebleeding is 0.0002 for initial grade 1, 0.0005 for initial grade 2, and 0.0009 for initial grade 3 (table 2). The posterior probability that the study-averaged probability of rebleeding is greater for initial grades 2 and 3 compared with grade 1 is 0.80 and 0.85, respectively. The posterior probability that the study-averaged probability of rebleeding is greater for initial grade 3 than for grade 2 is only 0.64. The posterior probability is 0.92 that initial grades 2 or 3 will suffer rebleeding as opposed to initial grade 1. Similar plots to figures 1⇑–3 are not shown for rebleeding because the occurrence rate was very low.
From the fit of the Bayesian random effects model we can conclude that the likelihood of retreatment is statistically significantly higher for aneurysms graded 2 or 3 than for those graded as 1 on the Raymond–Roy scale. Likewise, the difference between grades 2 and 3 is statistically significant for retreatment. The results also suggest that the likelihood of rebleeding tends to be higher for aneurysms with an initial grade higher than 1.
Discussion
The modified Raymond–Roy scale appears to provide some predictive value for aneurysm retreatment and rebleeding, but may be less predictive for more clinically relevant rates of recurrence of coiled cerebral aneurysms. In keeping with the individually reported results of the studies included in this cohort as well as other studies which were not included, there seems to be an overall trend toward higher rates of recurrence, rebleeding, and retreatment with higher initial Raymond–Roy grades. An initial coiling Raymond–Roy grade 3 or 2 has a significantly higher probability of increased rate of retreatment compared with grade 1. A significant difference was also demonstrated between initial coiling Raymond–Roy grades 3 and 2 for retreatment. Rebleeding rates were uniformly very rare, but we nonetheless found a higher probability of rebleeding for initial grades 2 or 3 compared with grade 1, which approached significance. Recurrence rates were not as reliably predictive using the Raymond–Roy grading system as were retreatment or rebleeding rates, although there was a trend toward higher recurrence with higher grade of initial occlusion. However, recurrence was defined as an increase in aneurysm after coiling, which may be more difficult to delineate with larger residual after initial coiling.
The fact that we fit a Bayesian random effects model instead of a frequentist model was largely for ease of interpretation. A random effects model fit via maximum likelihood estimation yielded a similar if not almost identical fit, as did the method of generalized estimating equations. The Bayesian approach allows a straightforward interpretation in terms of the probability of recurrence, retreatment, or rebleeding for one initial grade being higher or lower than another grade. Consequently, we report results using that model. Results from the frequentist analyses are available by request from the authors.
A random effects model assumes exchangeability among studies. Exchangeability implies that, prior to running each study, the studies could not be ordered on outcome based only on patient population or study design. This assumption applies to our model, but with the qualification that studies may have different effects of initial grade. Indeed, when we assumed initial grade effect to be constant across studies, the fit of the model was worse (in some cases, much worse) than when we allowed the effect of initial grade to vary across studies.
There are certainly a number of limitations to the results of our analysis. First, we did not use information regarding the time of recurrence, retreatment, or rebleeding. This information was not routinely provided in the cited articles, aside from the total follow-up time. The event time matters because a longer time to event is considered better than a shorter time to event. Second, the three-tiered modified Raymond–Roy scale, although probably more accurate than grading systems with more numerous divisions, is still subject to intra- and inter-observer variability2. Many of the studies were subjected to self-assessment bias without third party corroboration. Moreover, the decision to retreat or not retreat is rarely a simple decision based on recurrence—the clinical state of the patient as well as subjective assessment of the aneurysm morphology plays an enormous role in decision making. Furthermore, there was some variability in the follow-up imaging modality, although MR versus conventional angiography was usual. The type of retreatment, either surgical or endovascular, was also not fully addressed. As shown in table 1, there were few rebleed events in this meta-analysis, possibly due to aneurysm monitoring and retreatment. The low event rates may have limited the power for detecting differences between initial grades. More data are needed to confirm our observation that the rebleed rate appears to increase with initial grade.
Although commonly used in practice and research, most of the studies did not use a standardized metric for aneurysm treatment with very few using the Raymond–Roy grade for initial treatment and recurrence. Of our initial 4587 studies in the PubMed database search, only 10 met the basic criteria of using a standard grading system for describing initial and follow-up embolization results. Even then, some of the data in these 10 studies was not readily available in these terms and had to be tediously extrapolated from the body of the text. Moreover, we had to make more assumptions than we care to in evaluating the data.
Finally, we did not further categorize the results in our cohort with regard to initial clinical presentation, Hunt–Hess score, aneurysm size, location, morphology, coil selection (bare metal vs bioactive) and packing density, and adjunctive devices including balloons or stents. We expect that subgroup analysis of our data with respect to these parameters would have some effect on the results. Unfortunately, the publications in our cohort provide scant information with regard to these parameters—that is, while some of the publications may have addressed the clinical presentation of the patient or location of the aneurysms, the results of retreatment, recurrence, or rebleeding are not adequately categorized both with respect to the initial and follow up Raymond–Roy score as well as the initial clinical presentation and location of the aneurysm. Much of this information was not uniformly reported but may be further predictive of aneurysm recurrence, retreatment, and rebleeding. This may make for an argument to establish a national aneurysm registry in order to better analyze the data and improve predictability of long-term outcomes.
Conclusion
The Raymond–Roy grading system for aneurysm coiling appears to provide a predictive model for retreatment among the three grades. Our meta-analysis suggests that the Raymond–Roy grading system predicts aneurysm retreatment rates of 0.05 for grade 1, 0.11 for grade 2, and 0.23 for grade 3 initial occlusion. Our predicted retreatment rates were statistically significantly higher for higher initial aneurysm grades. Also, the probability of rebleeding was close to significantly higher for grades 2 or 3 versus grade 1. While there was an overall trend toward higher recurrence with higher grade of initial occlusion, the trends were not statistically significant. Similar results have been obtained using standard frequentist meta-analysis approaches. We conclude that the Raymond–Roy grading system has useful potential for predicting aneurysm events.
References
Footnotes
Contributors RD reviewed the study articles, compiled the data, drafted the initial analysis, and presented the preliminary data at SNIS conference. KC initiated the plan for the project, cleansed the obtained data, designed the data analysis plan, and revised the paper. LAT and ZZ instituted the complete statistical analysis plan, provided assistance in interpreting statistical data, designed and created data figures and charts, and offered revisions to the paper. All authors provided final approval.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Any additional data regarding this submission can be requested from the corresponding author via email.