Background We previously established three logistic regression models for discriminating intracranial aneurysm rupture status based on morphological and hemodynamic analysis of 119 aneurysms. In this study, we tested if these models would remain stable with increasing sample size, and investigated sample sizes required for various confidence levels (CIs).
Methods We augmented our previous dataset of 119 aneurysms into a new dataset of 204 samples by collecting an additional 85 consecutive aneurysms, on which we performed flow simulation and calculated morphological and hemodynamic parameters, as done previously. We performed univariate significance tests on these parameters, and multivariate logistic regression on significant parameters. The new regression models were compared against the original models. Receiver operating characteristics analysis was applied to compare the performance of regression models. Furthermore, we performed regression analysis based on bootstrapping resampling statistical simulations to explore how many aneurysm cases were required to generate stable models.
Results Univariate tests of the 204 aneurysms generated an identical list of significant morphological and hemodynamic parameters as previously (from the analysis of 119 cases). Furthermore, multivariate regression analysis produced three parsimonious predictive models that were almost identical to the previous ones, with model coefficients that had narrower CIs than the original ones. Bootstrapping showed that 10%, 5%, 2%, and 1% convergence levels of CI required 120, 200, 500, and 900 aneurysms, respectively.
Conclusions Our original hemodynamic–morphological rupture prediction models are stable and improve with increasing sample size. Results from resampling statistical simulations provide guidance for designing future large multi-population studies.
- Blood Flow
Statistics from Altmetric.com
Intracranial aneurysms affect 5–8% of the entire population.1 Aneurysm rupture leads to subarachnoid hemorrhage, a devastating event with high morbidity and mortality.2 Recent advancements in neurovascular imaging have increased the detection of asymptomatic unruptured aneurysms, placing more pressure on clinicians to decide which unruptured aneurysms to treat and which to observe, as treatments are fraught with complication risks and high costs. Currently, aneurysm size is the main quantitative discriminant used in evaluating rupture risk. However, small aneurysms still account for a large portion of ruptures.3 Consequently, shape based morphological parameters have been explored and correlated with rupture.3–5 On the other hand, hemodynamics are found to be associated with aneurysm rupture and play a fundamental role in the mechanisms of aneurysm rupture.6–12 Moreover, the recently released American Heart/Stroke Association guidelines for aneurysm management recommend that clinicians “consider morphological and hemodynamic characteristics of the aneurysm when discussing the risk of aneurysm rupture”.13
In a previous study of 119 aneurysms,8 we identified morphological and hemodynamic factors that discriminated ruptured from unruptured aneurysms and, through multivariate logistic regression analysis, built three aneurysm rupture probability models based on morphology only, hemodynamics only, and the combined parameters. High probability of rupture status was found to be associated with larger size ratio (SR) in the morphological model, lower aneurysm averaged wall shear stress (WSS), and higher aneurysm averaged oscillatory shear index (OSI) in the hemodynamic model, and all three in the combined model. In this follow-up study, we asked the following questions: (1) would these models be different if we increase the sample size and (2) how many samples are required to build stable statistical models? The objective of the current study was to evaluate the stability of these models by answering these questions.
We collected a new cohort of 85 aneurysms (18 ruptured; 67 unruptured) in 74 consecutive patients imaged at Millard Fillmore Gates Hospital, Buffalo, New York, between 2009 and 2010 after approval by the institutional review board of the University at Buffalo. Demographics (age, gender, location, and type) of the new cohort are shown in table 1. This dataset was consecutive with the 119 aneurysms in our previous study.8
Morphological and hemodynamic parameter extraction
Morphological and hemodynamic parameters for each aneurysm were calculated as previously described.5 ,8 Briefly, DICOM images were segmented at the three-dimensional region of interest, aneurysm lumen, and adjacent vessels. An inhouse Matlab code was used to calculate six morphological parameters5: aneurysm size, SR, aspect ratio (AR), Ellipticity Index, Non-Sphericity Index, and undulation index (UI). For computational fluid dynamics simulations, finite volume meshes of 0.5–1 million elements were imported into the computational fluid dynamics solver to calculate time resolved three-dimensional velocity and pressure fields. Three pulsatile cycles were simulated, with the last cycle being taken as output to ensure that numerical stability had been reached. All data presented were time averages over the third pulsatile cycle of flow simulation when applicable.
From the flow solutions, we calculated seven hemodynamic parameters described in detail previously8: WSS, maximum WSS (MWSS), low WSS area percentage (LSA), OSI, relative resident time (RRT), WSS gradient (WSSG), and number of vortices (NV). WSS is tangential frictional stress caused by blood flow on the vessel wall. In the statistical analysis, we averaged WSS over a cardiac cycle, and further averaged over the aneurysm sac. MWSS is the maximum time averaged aneurysmal WSS magnitude. LSA is defined as areas of the aneurysm wall exposed to WSS below 10% of the mean parent arterial WSS. OSI measures the direction change of WSS during the cardiac cycle, and is defined as aneurysm averaged OSI for quantitative analysis. RRT reflects the residence time of blood near the wall and is inversely proportional to the magnitude of the time averaged WSS vector. WSSG measures the change in WSS magnitude in the flow direction. NV is counted based on the velocity field of the representative cross sectional plane for each aneurysm. As with our original paper,8 for aneurysm averaged WSS, MWSS, and RRT, we normalized them by parent vessel average values.
Stability testing of the predictive models
To test the stability of our previous rupture prediction models,8 we aggregated the new (85 aneurysms) and original (119 aneurysms) cohorts into one dataset of 204 aneurysms. Univariate significant tests (Student's t test for normally distributed data or the Wilcoxon rank sum test for abnormally distributed data) of the 13 morphological and hemodynamic parameters identified significant parameters. The significant level p<0.01 was considered statistically significant with Bonferroni correction. Multivariate logistic regression using stepwise elimination was then applied to the significant morphological, hemodynamic, and combined parameters.8 The new multivariate logistic regression models were compared against the original models. We tested whether the new models were comprised of the same parameters. If so, we used the CI at the 95% level to examine the consistency of these two sets of models. Receiver operating characteristics (ROC) analysis was applied to compare the performance of the regression models through the area under the ROC (AUC-ROC) when applicable.
Resampling statistical simulation
In order to know how many aneurysm cases are required to generate sufficiently stable models for the benefit of future large population aneurysm rupture risk studies, we performed a simulation study for the logistic regression analysis based on the bootstrapping resampling statistical method to investigate the convergence of CI width of the coefficients in the regression models.14 This is conceptually similar to the grid convergence study commonly conducted in numerical simulations. Bootstrapping can assign measures of accuracy (eg, CIs) to sample estimates.14 It evaluates a variability of an estimator through resampling, assuming that the collected data have the same distributional properties as the original population. We carried out statistical simulations where the same set of variable entries was used in the logistic regression models (SR in the morphological model, WSS and OSI in the hemodynamic model, and all three in the combined model) in the following steps:
From the aggregated dataset of 204 aneurysms, we carried out random sample selection for increasing sample size n (n from 30 to 1000 in increments of 20). The case selection was random and thus some cases may have been selected multiple times.
At each step (n aneurysms), we randomly generated 1000 samples from the 204 aneurysms using bootstrapping replication. For each selection of n aneurysms, we performed logistic regression and calculated CI width for the coefficients of the regression model.
At each step, the process was repeated 1000 times to calculate the average of CI width.
CI width (each step giving lower and upper limits) and relative change (difference in interval width with n aneurysms minus interval width with n−20 aneurysms, divided by interval width with n aneurysms) were plotted and analyzed.
All statistical analysis was done using SPSS V.17.0 software (SPSS Inc, Chicago, Illinois, USA) and the R Project for Statistical Computing.
Figure 1 shows aneurysm geometry, flow streamlines, WSS distribution, and OSI distribution of four representative ruptured and four representative unruptured aneurysms from the new cohort. WSS distribution in figure 1 is the pointwise time averaged WSS distribution, while in our statistical analysis, WSS is further averaged over the aneurysm sac. Univariate testing of the 204 samples showed that aneurysm size and the hemodynamic factor WSSG were not discriminators of rupture at the significance level of 0.01, whereas all other parameters, including SR, UI, Non-Sphericity Index, Ellipticity Index, AR, WSS, MWSS, OSI, LSA, RRT, and NV, were significant at distinguishing ruptured from unruptured aneurysms (p<0.001). This finding is consistent with previous findings based on the original cohort of 119 aneurysms.8
Based on these significant parameters, multivariate logistic regression analysis of the aggregated cohort of 204 aneurysms generated three new risk stratification models: morphology only, hemodynamics only, and combined. The new hemodynamics only regression model consisted of WSS and OSI as independent predictors, which is consistent with the original hemodynamic model.8 However, the new morphology only model included both SR and UI, whereas in the original multivariate regression model only SR was significant.8
To investigate the contribution of UI to the morphological model, we used SR alone to build a univariate logistic regression model from the 204 samples and compared it against the model resulting from multivariate regression containing both SR and UI. The two models had very similar AUC-ROC values (0.831 and 0.835, respectively, figure 2). This indicates that the contribution of UI to classification of aneurysm rupture status through morphological regression models is minimal. We therefore chose the SR alone model as the parsimonious morphological predictive model from the 204 cases.
The final parsimonious predictive models based on the aggregated 204 aneurysms for morphology only, hemodynamics only, and combined are: 1 2 3
where Odds=p/(1−p) is the odds and p is the probability of an aneurysm being ruptured. Comparing Equations 1–3 from 204 aneurysms against the original Equations 4–6 from the 119 aneurysms in Xiang et al,8 we observe essentially the same three models with only slight differences in model coefficients. However, these coefficients have overlapping CIs for corresponding coefficients (figure 3). Evidently, when the sample size increased from 119 to 204, the CI width at the 95% level drastically decreased from 0.88 to 0.58 for SR in the morphological model; from 0.72 to 0.57 for WSS and from 3.18 to 2.19 for OSI in the hemodynamic model; and from 1.05 to 0.67 for SR, from 0.79 to 0.61 for WSS, and from 3.28 to 2.20 for OSI in the combined model. Because of the decreased variability (thus increased confidence), we suggest to use the updated rupture prediction models from the 204 samples (Equations 1–3), until they are replaced by future models extracted from larger datasets or with better performance.
Results by bootstrapping resampling simulations are given in figure 4. Figure 4A shows reduction in CI at the 95% level as the number of samples was increased, while figure 4B demonstrates the relative change in CI for the model coefficients. The model coefficients converged to their final values as more samples were added. We found that a level of 10%, 5%, 2%, and 1% CI convergences required 120, 200, 500, and 900 aneurysms, respectively. This information provided the insight for the future large sample and multi-center aneurysm rupture risk study.
Increasing detection of unruptured aneurysms places more and more pressure on neurosurgeons and neurointerventionalists to weigh rupture risk against surgical complication risks before making treatment decisions. Aneurysmal morphology and hemodynamics show great promises for rupture risk stratifications.3–11 ,15–18 The importance of aneurysmal morphology and hemodynamics for rupture risk assessment was also stressed by the American Heart and Stroke Association.13 In our previous study of 119 aneurysms, we found that SR, WSS, and OSI were independent predictors, and provided three regression models for aneurysm risk stratifications, based on hemodynamics only, morphology only, and hemodynamics and morphology combined. In the current study, we demonstrated the stability of the models by comparing regression models from an augmented sample of 204 aneurysms against those from the original 119 aneurysms. The three classification models were shown to be stable and, furthermore, improved with increasing sample size.
In the morphology category, we found that aneurysm size was not significant while AR was significant, which is consistent with findings from many other studies.3 ,5 In the final morphological model resulting from multivariate regression of the 204 samples, UI was also retained in addition to SR; however, ROC analysis indicated that the contribution of UI to the model was minimal. SR, a concept originally proposed by our group,5 ,19 has been found in many recent studies to be a significant predictor of aneurysm rupture status, regardless if SR was defined on a three-dimensional5 ,8 ,20–22 or two-dimensional basis,15 ,23 or which linear length was adopted to measure aneurysm ‘size’ in the ratio calculation.24 In a large study of 854 ruptured and 180 unruptured aneurysms, Kashiwazaki et al discovered that SR, but not the absolute aneurysm size, highly predicted rupture status in small aneurysms (<5 mm).20
In the hemodynamics category, we found that low WSS and high OSI were independently correlated with ruptured aneurysms, and that a model including these two parameters provided the odds of rupture. These were exactly the same findings as in our previous analysis of 119 aneurysms.8 Many studies have found a correlation between low WSS and ruptured aneurysms.4 ,6 ,7 ,25 ,26 Low WSS regions are typically located at the aneurysm dome,11 where 84% of ruptures occur.27 Low WSS and high OSI are related to ‘disturbed’ flow,28 which causes endothelial cells to decrease endothelial nitric oxide synthase activity, upregulates surface adhesion molecules, and increases endothelial permeability. All of these promote atherogenesis and inflammatory cell infiltration.28 Inflammation has been thought to be a key mechanism for intracranial aneurysm rupture.29–31 Our results provide further statistical evidence supporting the association of low WSS and high OSI with rupture identified previously.6–8
Nevertheless, a potential role of high WSS in aneurysm rupture should not be excluded,7 especially in small, conservatively followed aneurysms32 and aneurysms with jet impingement in the sac.33 ,34 High WSS resulting from flow impingement on the wall has been shown to trigger aneurysm degradation, as described in aneurysm initiation35 and progression.36 In the aneurysm cohort of the current study, a few of the ruptured aneurysms appeared to be dominated by impinging flow, but high WSS or high MWSS did not contribute significantly to the predictive models.
In light of the controversy and confusion surrounding whether low WSS or high WSS better predicts rupture, we have recently proposed a unified hypothesis that both low WSS and high WSS could be responsible for aneurysm growth and rupture via two independent hemodynamically driven biological pathways.6 ,7 However, it appears that more ruptured aneurysms are driven by the low WSS mechanism than the high WSS mechanism, based on many more reports of low WSS correlation with rupture8 ,10 ,11 ,17 ,18 ,25 ,26 ,37–40 than high WSS.9 ,33 ,41 ,42
This study has also investigated how many samples are required for building stable statistical models at different convergence levels of CI. We performed regression analyses using bootstrapping resampling of 30–1000 aneurysms from the aggregated pool of 204 aneurysms. We have demonstrated that an increasing level of CI convergence requires an increasing number of aneurysm samples. The resampling statistical simulation sheds light on future multicenter and multi-population studies. It provides guidance on the numbers of aneurysms needed to achieve a certain level of convergence for CI width. For example, in order to achieve a 1% convergence level for the CI width of the models, the target sample size should be approximately 1000 aneurysms.
This study has several limitations. First, our dataset may have a population bias, and hence our conclusions may not be valid for different patient populations. In the future, multicenter studies with larger multi-population datasets are needed to validate these models and may derive new models.43 Secondly, our current models are limited to image derived morphological and hemodynamic parameters. In the future, comprehensive aneurysm rupture risk statistical models should also consider other risk factors, including demographic, genetic, wall biomechanical, and medical factors. Thirdly, the rupture probability calculated from our predictive models does not involve time because, like most other aneurysm rupture risk studies, ours were based on cross sectional data and not on prospective longitudinal data. Finally, aneurysm geometries may have been affected by the rupture event, although increasing evidence indicates that aneurysms do not shrink when they rupture.5 ,44
The hemodynamic and morphological models for aneurysm rupture status prediction are stable and statistically significant. Augmenting the dataset improves the model coefficient estimation. Regression analysis from bootstrapping resampling statistical simulation sheds light on the design of future large and multicenter aneurysm rupture risk studies.
The authors thank Vincent M Tutino, MS, for help with the illustrations.
Contributors Conception and design: JX, AHS, and HM. Acquisition of the data: all authors. Analysis and interpretation of the data: JX, JY, AHS, and HM. Drafting the manuscript: JX and HM. Critical revision of the manuscript: all authors. Final approval of the manuscript: all authors.
Funding NIH grant-R01NS064592, a grant from Toshiba Medical Systems (to HM), and Dr Richard J Schlesinger grant from the American Society for Quality Biomedical Division (to JX).
Competing interests JX: Principal investigator of Dawn Brejcha Chair of Research grant from Brain Aneurysm Foundation. KVS: consultant for Toshiba; speakers’ bureau for Toshiba, ev3/Covidien, and the Stroke Group; and honoraria from Toshiba, ev3/Covidien, and the Stroke Group. EIL: shareholder/ownership interests in Intratech Medical Ltd, Mynx/Access Closure, and Blockade Medical LLC; principal investigator for Covidien US SWIFT PRIME trials; and other financial support from Abbott for carotid training for physicians. AHS: research grants as co-investigator of NIH grants (R01NS064592 and 5R01EB002873) and Research Development Award from the University at Buffalo; financial interests in Blockade Medical, Hotspur, Intratech Medical, Lazarus Effect, StimSox, and Valor Medical; consultant for Blockade Medical, Codman and Shurtleff Inc, Concentric Medical, ev3/Covidien Vascular Therapies, GuidePoint Global Consulting, Lazarus Effect, MicroVention, Penumbra, Stryker, and Pulsar Vascular; National Steering Committee for 3D Separator Trial (Penumbra Inc), SWIFT PRIME trial (Covidien), and FRED trial (MicroVention); speakers’ bureau for Codman and Shurtleff Inc; advisory board for Codman and Shurtleff Inc and Covidien Neurovascular; and honoraria from Abbott Vascular and Codman & Shurtleff Inc for training other physicians in carotid stenting and endovascular stenting for aneurysms, and Penumbra Inc. HM: principal investigator of NIH grant (R01NS064592), grant from Toshiba Medical Systems, and the Carol W Harvey Memorial Chair of Research grant from the Brain Aneurysm Foundation.
Ethics approval The study was approved by the institutional review board of the University at Buffalo.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.