Background and purpose We aimed to identify the best definition of early neurological improvement (ENI) at 2 and 24 hours after mechanical thrombectomy (MT) and determine its ability to predict a good functional outcome at 3 months.
Methods This retrospective analysis was based on a prospectively collected registry of patients treated by MT for ischemic stroke from May 2010 to March 2017. We included patients treated with stent-retrievers with National Institute of Health Stroke Scale (NIHSS) score before treatment and at 2 and/or 24 hours after treatment and modified Rankin Score (mRS) at 3 months. Receiver operating characteristic curve analysis was performed to estimate optimal thresholds for ENI at 2 and 24 hours. The relationship between optimal ENI definitions and good outcome at 3 months (mRS 0–2) was assessed by logistic regression.
Results The analysis included 246 patients. At 2 hours, the optimal threshold to predict a good outcome at 3 months was improvementin the NIHSS score of >1 point (AUC 0.83,95% CI 0.77 to 0.87), with sensitivity and specificity 78.3% (62.2–85.7%) and 84.6% (77.2–90.3%), respectively, and OR 12.67 (95% CI 4.69 to 31.10, p<0.0001). At 24 hours, the optimal threshold was an improvementin the NIHSS score of >4 points (AUC 0.93, 95% CI 0.89 to 0.96), with sensitivity and specificity 93.8% (87.7–97.5%) and 83.2% (75.7–89.2%), respectively, and OR 391.32 (95% CI 44.43 to 3448.35, p<0.0001).
Conclusions ENI 24 hours after thrombectomy appears to be a straightforward surrogate of long-term endpoints and may have value in future research.
Statistics from Altmetric.com
Early neurological improvement (ENI) corresponds to the improvement in the National Institute of Health Stroke Scale (NIHSS) score after a predetermined time. Various definitions of ENI have been described as predictors of favorable outcome after intravenous thrombolysis (IVT).1–3 However, this early endpoint has been adopted infrequently in studies of the endovascular treatment of acute ischemic stroke4 5 and has been used as a primary endpoint in only one recent single-center randomized trial.6 Because ENI is an early endpoint, it can isolate precisely the effect of mechanical thrombectomy (MT) itself, rather than the influence of subsequent care. We aimed to identify the best definition of ENI at 2 and 24 hours after MT and to determine whether it reliably predicted a good functional outcome at 3 months.
This retrospective analysis was based on a prospectively collected registry of consecutive patients treated by MT for ischemic stroke from May 2010 to March 2017. The present study complies with the Standards for Reporting of Diagnostic Accuracy and was approved by our local Ethics Committee, which waived informed consent due to the retrospective nature of the study.7 We included all patients who met the following inclusion criteria: (1) acute ischemic stroke treated by MT with stent-retrievers; (2) NIHSS score available before treatment, at 2 and/or 24 hours after treatment; and (3) modified Rankin Score (mRS) available at 3 months. We recorded demographic details, medical history, and clinical and imaging data.
In our institution, stroke neurologists perform multiple daily assessments of the NIHSS, including fixed assessment at 2 and 24 hours after MT for each patient. The baseline extent of the infarct was evaluated with the diffusion weighted imaging–Alberta Stroke Programme Early CT Score (DWI-ASPECTS). Recanalization was defined as a modified Thrombolysis in Cerebral Infarction (mTICI) score of 2b–3. Good functional outcome at 3 months (mRS 0–2) was evaluated by an independent neurologist.
After being tested for normality using the Shapiro–Wilk test, continuous variables were described as mean±SD or median (IQR). Receiver operating characteristics (ROC) curve analysis was performed to estimate the optimal thresholds for ENI at 2 hours and 24 hours. We determined optimal thresholds by maximizing the Youden index. Confidence intervals (CI) were calculated using 1000 bootstrap replications. Pairwise comparison of ROC curves was done to compare their accuracy. Additionally, we compared the thresholds we obtained with previously published definitions of ENI—that is, early clinical recovery (defined as NIHSS improvement ≥10 points) and dramatic recovery (defined as total NIHSS 0–3) at 2 and 24 hours.2 8 Variables were compared between patients with good and poor outcome at 3 months using the Mann–Whitney U test for continuous variables and the χ2 or Fisher’s exact test for categorical variables. We further assessed the relationship between optimal ENI at 2 hours and 24 hours and good neurological outcome at 3 months in separate multivariate binary logistic regressions models adjusted for previously published major clinical outcome predictors—namely, age, pretreatment NIHSS, serum glucose, type of treatment (bridging or thrombectomy alone), pre-therapeutic DWI-ASPECTS, and recanalization.9–11 P<0.05 was considered statistically significant. Statistical analyses were performed using Medcalc software (Version 14.12.0, Ostend, Belgium).
A total of 246 patients were included, with 87% treated under conscious sedation. NIHSS scores were available for 236 patients at 2 hours and for 244 patients at 24 hours. Two hours post-MT the optimal threshold to predict a good neurological outcome at 3 months was an improvement in the NIHSS score of >1 point (AUC 0.83 (95% CI 0.77 to 0.87); p<0.0001). This threshold provided sensitivity of 78.3% (69.2–85.7%), specificity of 84.6% (77.2–90.3%) (figure 1A) and OR of 12.67 (95% CI 4.69 to 31.10, p<0.0001). At 24 hours the optimal threshold was an improvement of >4 points (AUC 0.93 (95% CI 0.89 to 0.96); p<0.0001) with sensitivity and specificity 93.8% (87.7–97.5%) and 83.2% (75.7–89.2%) (figure 1B), respectively, and OR 391.32 (95% CI 44.43 to 3448.35, p<0.0001). Pairwise comparison of ROC curves revealed that ENI assessment had a stronger correlation with a good 3-month outcome when performed at 24 hours than at 2 hours (p<0.001). Based on these cut-offs, 43.6% of patients had ENI at 2 hours and 52.5% at 24 hours. Furthermore, 80.4% of patients with ENI at 2 hours and 82.8% of patients with ENI at 24 hours had good functional recovery at 3 months (figure 2). Regarding previous definitions in the literature, the diagnostic accuracy of ENI at 2 hours was superior to early clinical recovery (AUC 0.83 (0.77–0.87) vs 0.62 (0.55–0.68), p<0.0001) and dramatic recovery (AUC 0.83 (0.77–0.87) vs 0.59 (0.52–0.65), p<0.0001). Similarly, at 24 hours the diagnostic accuracy of ENI was superior to early clinical recovery (AUC 0.93 (0.89–0.96) vs 0.77 (0.72–0.83), p=0.002) and dramatic recovery (AUC 0.93 (0.89–0.96) vs 0.81 (0.76–0.81), p=0.0001).
Factors influencing ENI and good functional outcome at 3 months
ENI at 24 hours was influenced by several variables including clinical and imaging factors, time measures, collaterals, and recanalization status, as shown in table 1. ENI at both 2 and 24 hours was strongly correlated with good functional recovery at 3 months (both p<0.0001), both in univariate (online supplemental table 1) and multivariate analysis (table 2), with greater odds of a good outcome using ENI at 24 hours. As expected, patients with a good outcome were younger (p=0.006), had lower baseline NIHSS (p<0.0001), lower glycemia (p=0.0003), higher DWI-ASPECTS (p=0.0002), shorter clots (p=0.0002), shorter delay to treatment end (p=0.006), better collaterals (p=0.003), received more intravenous thrombolysis (p=0.009), and were more frequently recanalized (p<0.0001). At 2 and 24 hours, ENI (improvement >1 and >4 points, respectively), early clinical recovery (improvement ≥10 points) and dramatic recovery (total NIHSS 0–3) were more frequently observed in patients with a good functional outcome at 3 months (p<0.0001). Additionally, we found no difference in collateral status between patients with and without IVT (p=0.91) in patients with ENI.
Our analysis highlights the prognostic value of a simple early neurological assessment after MT to predict the 3-month clinical outcome. We found that an improvement of at least four points in the NIHSS score 24 hours after MT can accurately predict the 3-month outcome (AUC 0.93) with high specificity and sensitivity. Despite quite good diagnostic accuracy, the threshold of 1 point in NIHSS improvement 2 hours after MT seemed to be not relevant clinically because of the possibilities of inter-rater variability in the NIHSS assessment.12 13 Indeed, although being simple, quick and a robust stroke deficit scale, the NIHSS also contains items with poor reliability and redundancy such as the level of consciousness, facial palsy, limb ataxia, and dysarthria.9 However, in our study the correlation between ENI and 3-month outcome was clearly stronger for an assessment performed at 24 hours.
The predictive power of ENI has been widely evaluated in large cohorts of patients treated with IVT. Unlike our study, NIHSS changes at 2 and 24 hours showed fair accuracy (AUC 0.65–0.75) for predicting final outcome.1 2 These studies underlined that no single definition of ENI provides optimal values for both specificity and sensitivity1; indeed, the earlier ENI occurred, the higher specificity and the lower sensitivity in general.2 The role of ENI in patients treated with MT has been poorly studied. In line with our results, an International Management Stroke III post hoc analysis (where 50% of patients were treated endovascularly) found that the evaluation of NIHSS at 24 hours showed a high level of accuracy (AUC 0.91) in predicting the final outcome.5 We hypothesize that such differences in the final outcome prediction may result from different kinetics of neurological recovery after MT or IVT. First, MT provides higher rates of rapid recanalization with subsequent early clinical improvement,14 while chemical lysis of the thrombus may be more progressive in large vessel occlusions. This may allow a higher amount of DWI reversal, a factor that strongly influences ENI.9 10 Second, in studies evaluating ENI after IVT, patients did not always have a large vessel occlusion whereas our patients had large vessel occlusions and higher baseline NIHSS scores. Although indicating a poor prognosis, higher scores also allowed a more significant decrease after early recanalization, meaning a higher chance of achieving ENI. Indeed, in a population of patients treated with IVT, Hemmen et al reported better accuracies of ENI while baseline NIHSS scores increased.1
Another important goal of our study was to define the optimal time point to determine if the patient shows ENI. We found that assessment at 24 hours was more valuable at predicting final outcome than at 2 hours. It is widely acknowledged that the NIHSS score often changes during the first 24 hours, and that the relationship between the neurological examination and final functional outcome strengthens after the initial hours and then begins to plateau.15 Indeed, in some recanalized patients a ‘stunned brain’ phenomenon (ie, no response to recanalization during a few hours) with delayed recovery can be observed.16 Consequently, we hypothesize that NIHSS testing at 2 hours may have been influenced by greater variability. Also, because MT necessitates anesthetic support to prevent patient movement, a very early neurological assessment may be impacted due to altered levels of consciousness, especially in patients treated under general anesthetic. However, in our study, conscious sedation was always offered as a first-line option, thus limiting the number of patients treated under general anesthesia. But a consequence is also that the impact of general anesthesia on 2-hour and 24-hour NIHSS assessment is difficult to determine in the present study, so our ENI thresholds need further validation cohorts of patients treated under general anesthesia.
While ENI seems to be a practical tool, a concern is whether it has the same meaning in patients with low and high NIHSS. We found a greater proportion of patients reaching ENI at 2 and 24 hours with increasing baseline NIHSS (p for trend=0.14 and 0.02, respectively). Higher baseline NIHSS allowed greater improvement of the score after early recanalization. However, this was counterbalanced by the smaller proportion of patients with a high baseline NIHSS score achieving a good functional outcome. Finally, we did not observe better accuracy of ENI while baseline NIHSS scores increased (online supplemental table 2). Additionally, we found that ENI seems to be a practical tool for routinely following patients with a high baseline NIHSS score, while using total NIHSS may be preferable for patients with a low baseline NIHSS score (online supplemental table 3).
The role of IVT combined with MT (bridging therapy) remains a matter of debate. Indeed, some studies comparing patients treated by bridging versus MT alone suggested similar clinical outcomes, revascularization, symptomatic intracranial hemorrhage, and mortality rates.17–19 In contrast, other studies with the same design suggested that prior IVT facilitates successful recanalization by MT.20–22 This suggests the pressing need for a randomized trial to compare direct MT with bridging therapy. In this situation of clinical uncertainty, we noted in our population a higher proportion of patients with IVT in the ENI group (p=0.02). One possible explanation is that successful arterial recanalization does not guarantee by itself good reperfusion, as distal blood flow may be compromised and not accessible to mechanical treatment but only to the synergistic effect of IVT. While waiting for a randomized trial comparing IVT+MT with MT alone, IVT should be started as early as possible for all eligible patients with stroke without delaying access to MT.
Our study has potential limitations. First, the single-center design may weaken the generalizability of our results. Second, despite an honorable sample, some analyses lacked precision, with wide CI. Ideally, larger studies are needed to confirm the clinical utility of ENI in patients treated by MT. Third, we did not test definitions of ENI based on percent change in NIHSS score, as has been done previously.2 4 23 24 However, we obtained significant results with absolute NIHSS changes which are more pragmatic. Our study is affected by some limitations of retrospective analyses of registry data, especially bias due to lack of blinding to treatment. Indeed, the NIHSS assessment was performed by stroke neurologists not blinded to the result of MT, but we believe this was unlikely to introduce a measurement bias because they were not aware of the study at that time. On the other hand, it is impossible to determine whether knowledge of the baseline NIHSS score may have influenced the neuroradiologist in some way during MT; however, the neuroradiologists were similarly not aware of the study at that time. Also, we did not assess the accuracy of the final infarct volume compared with ENI in the prediction of 3-month functional outcome because we focused on the analysis of pre-therapeutic factors affecting ENI. As the final infarct volume is a strong predictor of the final outcome,25 a future study comparing the ROCs of final infarct volume with ENI at 2 and 24 hours will be meaningful. Finally, we did not evaluate whether combining ENI with other variables available at 24 hours (eg, hemorrhagic transformation, infarct volume) could potentiate the final outcome prediction.
While a consideration of all possible pre-stroke and post-stroke variables in a predictive model might be ideal, it would likely be highly complex and the resulting tool impractical to use in clinical settings. Instead, it is possible that a complex variable such as ENI, that is influenced by several factors,23 26 27 could serve as a single determinant of clinical outcome.28 29 ENI at 24 hours after MT could provide a straightforward and reliable substitute for long-term endpoints in future research.
Contributors All authors have made a substantial contribution to the conception and design of the studies and/or the acquisition and/or the analysis of the data and/or the interpretation of the data; drafted the work or revised it for significant intellectual content; approved the final version of the manuscript; and agree to be accountable for all aspects of the work, including its accuracy and integrity.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial, or not-for-profit sectors.
Competing interests LP is consultant for Balt, Microvention, Neuravi, and Penumbra.
Patient consent Obtained.
Ethics approval Ethics approval was obtained from Reims Ethics Committee.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.