Background Benchmarking of complications is necessary in the context of the developing path to accountable care. We attempted to create a predictive model of negative outcomes in patients undergoing cerebral aneurysm coiling (CACo).
Methods We performed a retrospective cohort study involving patients who underwent CACo from 2005 to 2009 and who were registered in the Nationwide Inpatient Sample database. A model for outcome prediction based on individual patient characteristics was developed.
Results Of the 10 607 patients undergoing CACo, 6056 presented with unruptured aneurysms (57.1%) and 4551 with subarachnoid hemorrhage (42.9%). The respective inpatient postoperative risks were 0.3%, 5.7%, 1.8%, 0.8%, 0.5%, 0.2%, 1.9%, and 0.1% for unruptured aneurysms, and 13.8%, 52.8%, 4.9%, 36.7%, 1%, 2.9%, 2.3%, and 0.8% for ruptured aneurysms for death, unfavorable discharge, stroke, treated hydrocephalus, cardiac complications, deep vein thrombosis, pulmonary embolism, and acute renal failure. Multivariate analysis identified risk factors independently associated with the above outcomes. A validated model for outcome prediction based on individual patient characteristics was developed. The accuracy of the model was estimated by the area under the receiver operating characteristic curve, and it was found to have good discrimination.
Conclusions The presented model can aid in the prediction of the incidence of postoperative complications, and can be used as an adjunct in tailoring the treatment of cerebral aneurysms.
Statistics from Altmetric.com
Since the publication of the International Study for Aneurysm Treatment,1 there has been a paradigm shift in the treatment of cerebral aneurysms in the USA.2 ,3 Cerebral aneurysm coiling (CACo) has surfaced as the predominant treatment modality, not only for subarachnoid hemorrhage (SAH), but also for elective cases, increasing the total number of aneurysms treated in recent years.2 This trend for more coiling of unruptured aneurysms is not based on randomized controlled trials (RCT). Even in cases of SAH, where several RCT1 ,4 ,5 have supported the superiority of coiling at least after 1 year of follow-up (with the Barrow Ruptured Aneurysm Trial trial demonstrating equivalence in 3 years), the rigorous inclusion criteria restrict their results in certain patients with low operative risk and predefined age range. Determination of an estimated risk of adverse events for each individual patient could tailor the application of the results of evidence based medicine and assist in decision making, especially in patients suitable for clipping and coiling. This strategy can also allow the identification of modifiable risk factors associated with postoperative medical complications in patients undergoing coiling.
Several studies have attempted to identify such complications. However, they have mostly focused on comparing those for patients undergoing clipping versus coiling.6–13 There has been no investigation of modifiable patient level risk factors that can affect outcomes. Most of the studies have been retrospective analyses of single institution experiences,14 ,15 demonstrating results with limited generalization given their inherent selection bias. Interpretation of other multicenter studies is equally limited given their focus on specific subgroup data16–18 or their consideration of all morbidity as one variable.19
The Nationwide Inpatient Sample (NIS)20 is a hospital discharge database that represents approximately 20% of all inpatient admissions to non-federal hospitals in the USA. It allows for the unrestricted study of the patient population in question. Using this database, preoperative comorbidities associated with postoperative death, unfavorable discharge, stroke, treated hydrocephalus, cardiac complications, deep vein thrombosis (DVT), pulmonary embolism (PE), acute renal failure (ARF), and prolonged length of stay in patients undergoing CACo were identified. Based on these data, a risk factor based predictive model for negative outcomes in CACo was developed.
Nationwide Inpatient Sample database
All patients undergoing CACo in the NIS20 database (Healthcare Cost and Utilization Project, Agency for Healthcare Research and Quality, Rockville, Maryland, USA) between 2005 and 2009 were included in the analysis. The NIS is a prospective hospital discharge database that represents approximately 20% of all inpatient admissions to non-federal hospitals in the USA. For these years, the NIS contains discharge data regarding 100% of discharges from a stratified random sample of non-federal hospitals in some states to approximate a representative 20% subsample of all non-federal US hospital discharges. More information about the NIS is available at http://www.ahcpr.gov/data/hcup/nisintro.htm.
In order to establish the cohort of patients, we used the International Classification of Diseases, ninth revision, Clinical Modification (ICD-9-CM) codes to identify patients in the registry who underwent coiling (ICD-9-CM code 39.52 (should also have a code 88.41 and no 39.51 during the same hospitalization), 39.72, and 39.79) for ruptured (ICD-9-CM code 430, excluding 094.87 for ruptured syphilitic aneurysm, 437.4 for cerebral arteritis, 747.81 for arteriovenous malformation, 800.0–801.9, 803.0–804.9, 850.0–854.1, and 873.0–873.9 for traumatic hemorrhage, 39.53 or 92.30 for treatment diagnosis for arteriovenous malformation repair or radio surgery) and unruptured (ICD-9-CM code 437.3) cerebral aneurysms between 2005 and 2009 (figure 1).
The primary outcome variables (see online supplementary file) were the inpatient postoperative risks of stroke, cardiac complications, death, treated hydrocephalus, DVT, PE, ARF, length of stay more than 2 days for patients with unruptured aneurysms, length of stay more than 14 days for patients with ruptured aneurysms, and unfavorable discharge (transfer to short term hospital, skilled nursing facility, intermediate care, or rehabilitation) for patients registered in the NIS undergoing CACo. The selection of 14 days as the cut-off was based on the duration of the typical vasospasm window. It was used for the purpose of creating a predictive model and it did not signify inappropriately long hospitalization. The effect on the outcomes of the pertinent exposure variables (see online supplementary file) was examined in a multivariate analysis. Age was the only continuous variable. Gender, diabetes mellitus, chronic obstructive pulmonary disease, hypertension, hypercholesterolemia, peripheral vascular disease (PVD), congestive heart failure (CHF), coronary artery disease (CAD), history of stroke, obesity, chronic renal failure (CRF), and coagulopathy were categorical variables.
The multivariate logistic regression parameter values were used to construct a predictive model for postoperative complications. The C index (area under the receiver operating characteristic curve) was used to measure discriminatory ability. A value of 0.5 indicates a model of no discriminatory ability at all (not better than chance). A C index corrected for overfitting bias was calculated using leave out cross validation (eg, leave 10% out, repeated 1000 times, as well as bootstrap validation, repeated 1000 times). The Hosmer–Lemeshow test was used to assess calibration of the model. Interactions were tested but none was significant at the threshold (0.001) we set to correct for multiple testing (20×19/2=190 interactions in all) and none improved the C index by more than 0.002. Non-linear functions of the continuous characteristics (age) were explored using regression splines but none improved the model. Statistical analyses were performed using the Stata V.12, the XLSTAT V.2011.2.01 (Adinsoft), and R (V.2.15.1).
Demographics and clinical characteristics
In the selected study period there were 10 607 patients undergoing CACo (mean age 55.8 years; 71.6% women) who were registered in NIS (figure 1), of whom 6056 presented with unruptured aneurysms (57.1%) and 4551 with SAH (42.9%) (table 1).
The inpatient postoperative incidence of death, unfavorable discharge, stroke, treated hydrocephalus, cardiac complications, DVT, PE, ARF was 0.3%, 5.7%, 1.8%, 0.8%, 0.5%, 0.2%, 1.9%, and 0.1% for unruptured aneurysms, and 13.8%, 52.8%, 4.9%, 36.7%, 1%, 2.9%, 2.3%, and 0.8% for ruptured aneurysms, respectively (table 2). All outcomes, except cardiac complications and PE, were encountered significantly more frequently in cases of SAH.
Multivariate analysis of inpatient postoperative outcomes
A multivariate analysis investigating the effects of several risk factors on the risks of inpatient death (figure 2A), stroke (figure 2B), treated hydrocephalus (figure 2C), unfavorable discharge (figure 2D), length of stay more than 2 days (figure 2E), cardiac complications (figure 3A), ARF (figure 3B), DVT (figure 3C), and PE (figure 3D) for patient undergoing CACo for unruptured aneurysms was performed. A similar analysis for patient undergoing CACo for ruptured aneurysms was also performed (A–E and 5B–D).
In our model, we observed that history of stroke, CRF, and CHF were associated with higher mortality in all patients. Coagulopathy and increasing age were preferentially associated with worse outcomes in SAH. In addition, patients with SAH had a higher incidence of periprocedural stroke, which was most commonly seen in patients with a prior history of ischemic stroke. Older patients with prior ischemic stroke or cardiac disease appeared to have an increased rate of severe hydrocephalus, requiring intervention.
Increasing age and previous heart disease were associated with cardiac complications whereas prior heart disease, PVD, and coagulopathy were common factors associated with ARF. Patients who underwent longer immobilization (obesity, prior stroke) had a higher incidence of DVT and PE. Length of stay of more than 14 days in SAH patients, or more than 2 days in patients with unruptured aneurysms, was associated with increasing age, obesity, prior stroke, CHF, CAD, diabetes, and coagulopathy. Unfavorable discharge was, as expected, more commonly seen in SAH patients and was associated with increasing age, general medical comorbidities, prior stroke, and coagulopathy.
Predictive model application
The predicted risk for unfavorable discharge for unruptured and ruptured aneurysms, and for death, and length of stay longer than 14 days for ruptured aneurysms was plotted against age for the most significant individual variables, as well as their combinations (figure 6). An additive effect of the variables was observed on the risk of all four outcomes. The presence of coagulopathy in patients with unruptured aneurysms and prior history of stroke doubled the possibility of unfavorable discharge after CACo. A similar effect was the presence of CRF in patients with a history of stroke and ruptured cerebral aneurysms. In addition, history of coagulopathy and prior stroke doubled the risk of death in patients with ruptured aneurysms and CRF undergoing coiling. Finally, history of CAD and prior stroke in patients with SAH and CRF undergoing coiling increased their chance of hospitalization longer than 2 weeks by one-third.
Accuracy of the model
The area under curve or C statistic of the receiver operating characteristic curve was used to estimate the accuracy of the model. In patients with unruptured aneurysms, the bootstrap validated C index was calculated to be 0.78, 0.75, 0.83, 0.75, 0.77, 0.95, 0.63, 0.91, and 0.61 for postoperative risk of death, unfavorable discharge, stroke, treated hydrocephalus, cardiac complications, DVT, PE, ARF, and length of stay longer than 2 days, respectively, demonstrating good discrimination. In patients with ruptured aneurysms, this was calculated to be 0.68, 0.73, 0.70, 0.60, 0.77, 0.67, 0.65, 0.80, and 0.64 for postoperative risk of death, unfavorable discharge, stroke, treated hydrocephalus, cardiac complications, DVT, PE, ARF, and length of stay longer than 14 days, respectively, demonstrating good discrimination. The cross validated C index gave almost identical results as the above for both ruptured and unruptured aneurysms. Each model had good calibration, as assessed by a Hosmer–Lemeshow statistic (see online supplementary file).
Identification of modifiable risk factors associated with complications as well as the development of predictive models for outcomes are significant milestones in the path to accountable care.21 Especially in patients amenable to both clipping and coiling, individualized complication prediction can tailor decision, making for the appropriate intervention. The NIS provides diverse data representative of the national experience, and is not restricted by the rigorous inclusion criteria of RCTs for SAH, allowing the generalization of its conclusions. In addition, in the area of unruptured aneurysms, where RCTs are lacking, it can assist in the decision making for individual patients. Previous studies of the NIS have focused on comparing the morbidity and mortality of the two interventions,6 ,9–13 or the effect of volume on outcomes,19 avoiding any analysis of modifiable patient level risk factors.
History of stroke, CRF, and CHF was associated with higher mortality in all patients. Coagulopathy and increasing age were, as expected, preferentially associated with worse outcomes in SAH. The paradoxically protective effect of obesity and CAD might be associated with the potentially beneficial role of statins in the survival of patients with SAH.22 Alternatively, it might be attributed to the potential random effect of multiple comparisons. The observed inpatient mortality (0.3% for unruptured aneurysms and 13.8% for SAH) is slightly less than what is observed in the literature.6–10 14–17 ,19 ,23 Several prior studies on NIS have focused on the effect of increasing age on higher mortality.7 ,8
Morbidity immediately related to the surgical intervention (postoperative stroke and treated hydrocephalus) was also investigated. Patients with SAH had a higher incidence of periprocedural stroke. It was most commonly seen in patients with a prior history of ischemic stroke, probably due to their higher atherosclerotic burden. Treated hydrocephalus was, as expected, much higher in patients with SAH, given the high ventriculostomy rate. Patients with a heavier atherosclerotic burden (with prior ischemic stroke or cardiac disease) or limited reserve because of older age, appeared to have an increased rate of severe hydrocephalus. A potential explanation is that these patients have a higher Hunt and Hess grade at presentation and more complex operations, increasing the risk of hydrocephalus. The observed protective effect of hypercholesterolemia could again be associated with a beneficial effect of the use of statins on this outcome.24
The observed incidence of general medical morbidity measures was similar in all patients undergoing CACo, except for the higher rate of DVT and ARF in patients with SAH. The lack of anticoagulation and prolonged immobilization in this group of patients can explain the higher incidence of DVT. Increasing age and previous heart disease were associated with cardiac complications whereas heavier atherosclerotic burden (heart disease, PVD) and coagulopathy were common factors associated with ARF. Patients who underwent longer immobilization (obesity, prior stroke) had a higher incidence of DVT and PE. These rates are in accordance with what has been previously reported.6 ,10
Finally, we investigated length of hospitalization and disposition, which have a central role in the developing model of accountable care.21 Length of stay of more than 14 days in SAH patients, or more than 2 days in patients with unruptured aneurysms, was associated with increasing age, patient immobility (obesity and prior stroke), general medical comorbidities (CHF, CAD, and diabetes), and coagulopathy. Unfavorable discharge was, as expected, more commonly seen in SAH patients and was associated with increasing age, general medical comorbidities, and other factors (prior stroke, coagulopathy) that also contributed to prolonged hospitalizations. The effect of increasing age in unfavorable discharge has also been encountered in prior studies.7 ,8 Although the rate of unfavorable discharge in patients with unruptured aneurysms might appear high (5.7%), it can be placed in context if the rates of neurologic and other complications, which can result in extended post-acute care, are added.
The proposed predictive model for all outcomes was validated and demonstrated moderate accuracy and discrimination. It may provide a useful tool for patient counseling and informed decision making in the preoperative evaluation of patients undergoing CACo, especially in patients suitable for both interventions. This model can be further refined and validated in prospective studies of the US population. If the risk of morbidity and mortality is predicted to be unacceptably high for coiling, after optimizing any modifiable risk factors, alternative options should be sought after more aggressively (clipping, observation), if appropriate.
The present study has some limitations common to administrative databases. Indication bias and residual confounding could account for some of the observed associations. In addition, several coding inaccuracies can affect our estimates, as in other studies involving the NIS. However, coding for SAH has shown nearly perfect association with medical record review.25 The NIS during the years studied did not include hospitals from all states.20 However, the hospitals included were still diverse with respect to size, region, and academic status, supporting the generalizability of our findings. The NIS does not provide any clinical information on the structure, size, or location of the aneurysms, which are important factors to be considered in cerebrovascular neurosurgery. Additionally, we were lacking post-hospitalization and long term data on these patients, as well as disease severity in patients presenting with SAH.
There is also a potential bias in the patient level risk factors selected to be investigated. Further, there is a bias in the assignment of the length of prolonged hospitalization (more than 14 and 2 days for rupture and unruptured aneurysms, respectively). However, these time limits were decided based on the most common values presented in the literature,9–11 and are only serving the purpose of creating a predictive model. Finally, the retrospective nature of the study introduces a selection bias that can affect the outcomes of coiling.
The NIS is a prospective hospital discharge database that contains a representative sample of all inpatient admissions to non-federal hospitals in the USA. By using it, we identified independent risk factors associated with death, unfavorable discharge, stroke, treated hydrocephalus, cardiac complications, DVT, PE, ARF, and prolonged hospitalization in CACo patients. Based on the results of the multivariate analysis, a risk factor based predictive model of all the outcomes was devised and validated. Although the generalization of these predictions should be done with caution, the model can potentially assist with risk stratification and tailoring of surgical decision making in individualized patients. This is particularly important in the context of the increased use of non-risk adjusted metrics of performance for benchmarking. As complex cases are more frequently performed at tertiary centers, the absence of risk adjustment would create a bias against comprehensive stroke centers where most of these patients are treated.
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Data supplement 1 - Online supplement
KB and SM contributed equally and are co-primary authors.
Contributors KB: conception and design, data analysis, manuscript preparation, and study supervision. SM: data collection and analysis, and manuscript review. TAM: data analysis and manuscript review. AF: data collection and manuscript review. NL: data interpretation, conception, study supervision, and manuscript review. CE: study supervision and manuscript review.
Competing interests None.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.