Article Text
PDF
Abstract
Background and purpose Our aim was to revalidate the RACE scale, a prehospital tool that aims to identify patients with large vessel occlusion (LVO), after its region-wide implementation in Catalonia, and to analyze geographical differences in access to endovascular treatment (EVT).
Methods We used data from the prospective CICAT registry (Stroke Code Catalan registry) that includes all stroke code activations. The RACE score evaluated by emergency medical services, time metrics, final diagnosis, presence of LVO, and type of revascularization treatment were registered. Sensitivity, specificity, and area under the curve (AUC) for the RACE cut-off value ≥5 for identification of both LVO and eligibility for EVT were calculated. We compared the rate of EVT and time to EVT of patients transferred from referral centers compared with those directly presenting to comprehensive stroke centers (CSC).
Results The RACE scale was evaluated in the field in 1822 patients, showing a strong correlation with the subsequent in-hospital evaluation of the National Institute of Health Stroke Scale evaluated at hospital (r=0.74, P<0.001). A RACE score ≥5 detected LVO with a sensitivity 0.84 and specificity 0.60 (AUC 0.77). Patients with RACE ≥5 harbored a LVO and received EVT more frequently than RACE <5 patients (LVO 35% vs 6%; EVT 20% vs 6%; all P<0.001). Direct admission at a CSC was independently associated with higher odds of receiving EVT compared with admission at a referral center (OR 2.40; 95% CI 1.66 to 3.46), and symtoms onset to groin puncture was 133 min shorter.
Conclusions This large validation study confirms RACE accuracy to identify stroke patients eligible for EVT, and provides evidence of geographical imbalances in the access to EVT to the detriment of patients located in remote areas.
- stroke
- thrombectomy
- thrombolysis
Statistics from Altmetric.com
Introduction
The beneficial effects of endovascular treatment (EVT) have been robustly proven in patients with acute ischemic stroke due to large vessel occlusion (LVO).1 Because of the limited availability of EVT and its time-dependent efficacy, there is a critical need to develop strategies to identify at a prehospital level those patients who are candidates for EVT.2
In the past few years, several prehospital scales have been proposed. The optimal clinical tool should strike the right balance between sensitivity and specificity, that is, its ability to capture as many LVO patients as possible to avoid an excess of unnecessary transfers to comprehensive stroke centers (CSC). Although there is no evidence from randomized clinical studies about the best transfer models for patients with suspected LVO, several guidelines include triage algorithms using these scales to select patients that may be directly transferred to a stroke center with EVT capacity.3 4
Out of these prehospital scales, only the RACE scale has been prospectively validated in the field using a cohort of patients with a suspicion of an acute stroke in a limited area of Catalonia,5 and large studies demonstrating the real-life accuracy of LVO scales are lacking. Here, we aimed to revalidate the RACE scale as a prehospital tool to identify both patients harboring LVO patients and patients eligible for EVT after its implementation in the territorial stroke code protocol throughout Catalonia. As a secondary objective, we aimed to analyze geographical differences regarding access to the EVT of patients with suspected LVO in a territorial network organized following a drip-and-ship model.
Methods
Study setting
The region of Catalonia includes a total population of 7.5 million inhabitants and an area of 32 000 sq km. In 2006 the Catalan Stroke Program established a stroke code system for the entire territory of Catalonia, covered by 20 primary stroke centers, 12 telestroke centers, and six CSC capable of providing EVT. The stroke code is activated by emergency medical services (EMS; 65%) or the emergency department of any hospital (35%) in patients without prior disability with clinical suspicion of acute stroke within 8 hours of symptoms or when the time from onset is uncertain. Every year in Catalonia there are about 6000 stroke code activations, and 15% of them are stroke mimics.
Care of patients included in this study occurred accordingly to a drip-and-ship model in use throughout Catalonia, before the start of the RACECAT trial (March 2017) that compares mother ship vs. drip-and-ship for LVO suspected stroke patients. Accordingly, patients with a suspected acute stroke cared for EMS with a suspected acute stroke were transferred to the nearest stroke center regardless of hospital type and patient’s characteristics, where intravenous thrombolysis (iv tPA) was administered if eligible. Candidates to EVT were secondarily transferred from referral, primary or telestroke centers to a CSC if vascular imaging demonstrates a LVO or, if unavailable, based on clinical data (National Institute of Health Stroke Scale (NIHSS) ≥6 with cortical signs).
The RACE scale is a simple and rapid neurological scale designed to detect acute stroke patients with a high probability of having an LVO and therefore of being eligible for endovascular therapy (www.racescale.org). The RACE scale is a simplification of the NIHSS scale using those items with a higher ability to predict the presence of a LVO, and evaluates five items: facial palsy, brachial paresis, crural paresis, oculocephalic deviation, and aphasia/agnosia, with a total score of 0–9. Design methods and results of the validation study have been previously published.5 Briefly, a RACE ≥5 scored by EMS predicted LVO with a sensitivity of 85% and specificity of 69%. The RACE scale evaluation was then added to the stroke code protocol of Catalonia in September 2014, after training all EMS professionals (>3000 technicians, nurses, and physicians of the EMS) throughout a 4-hour certified online course (http://racescale.org/2017/11/06/curso-online-new/).
Data registry
This observational study is based on data available in the CICAT registry, a government-mandated, prospective, hospital-based dataset that includes all stroke code activations. CICAT is linked to the EMS database to capture information about prehospital care, including the times workflow (time from onset, call to EMS, EMS first attention, EMS hospital arrival), and clinical data as vital signs and RACE scale recorded by EMS. At a hospital level, the following variables are registered: stroke subtype (intracerebral hemorrhage, ischemic stroke, transient ischemic attack (TIA) or stroke mimic); NIHSS at admission; the presence of LVO on admission, site of occlusion, and diagnostic method (Computed Tomography Angiography (CTA), Magnetic Resonance Angiography (MRA) or transcranial duplex); and revascularization treatment (none, iv tPA alone, iv tPA plus EVT or EVT alone). The following times metrics are registered: time from onset (missing in wake-up stroke or if time of symptoms onset is unknown); EMS alert; first hospital arrival (door-in); iv tPA bolus administration, start of secondary transfer in referral centers (door-out); and CSC arrival, groin puncture, and arterial recanalization with EVT.
For this study, we used data from consecutive patients in whom the stroke code was activated directly by EMS from during a total of 9 months.
The governmental regional CICAT registry satisfies all legal requirements mandated by the local law of personal data protection. For that reason, local Ethical Committee evaluation was not necessary for this analysis.
Outcome measures and statistical analysis
The primary variable was the presence of a LVO, defined as an occlusion of terminal intracranial carotid artery (TICA), proximal segment of the middle cerebral artery (MCA-M1), tandem (extracranial carotid artery plus middle cerebral artery), and basilar artery. Accuracy analysis was also performed for a broader definition of LVO including distal segment of MCA (MCA-M2). Patients with acute ischemic stroke and no information about vascular status were excluded for the accuracy analysis to detect LVO. Patients with final diagnosis of transient ischemic attack, intracerebral hemorrhage, and stroke mimic were classified as having no LVO. Outcome measures were the rate of EVT and time from onset to EVT.
The predictive capacity of the RACE scale to identify patients with LVO and patients receiving EVT was evaluated with receiver operating curve (ROC) analysis and the area under ROC (c-statistics). Ideal prediction produces a c-statistic of 1.00;: precision no better than chance is associated with c-statistic of<=0.50. Sensitivity, specificity, positive predictive value (PPV), negative predictive values, and overall accuracy to identify LVO and patients receiving EVT were calculated for the pre-established cut-off RACE ≥5. Correlation between RACE scores measured by EMS at a prehospital setting and NIHSS scores measured at hospital admission by neurologists, was analyzed with the non-parametric Spearman coefficient. Characteristics of false positive patients (RACE ≥5 with no LVO) and false negative patients (RACE <5 and LVO) were compared with patients correctly identified (true positive, RACE ≥5 with LVO) in a bivariate analysis.
For the secondary objective, patients were classified into two groups according to their location: area directly covered by a CSC (six centers); and covered by a referral center (20 centers). The proportion of patients with a suspicion of LVO (RACE ≥5) treated with EVT and time from onset to EVT were compared using the Chi-square test and non-parametric Kruskal–Wallis tests, respectively. A multimodal analysis adjusted by stroke severity and age was performed to compare the odds of receiving EVT, considering the group 1 (directly covered by a CSC) as reference.
Results
We included in this study 1822 patients out of 2378 consecutive EMS stroke code activations, in whom the RACE scale was evaluated (76.6%). Baseline characteristics of included and excluded patients are shown in (online supplemental appendix). Patients without an EMS assessment of the RACE scale were more frequently minor stroke and TIAs. Stroke subtype distribution of included patients was ischemic stroke in 1110 patients (61.0%), intracerebral hemorrhage in 320 (17.6%), TIA in 101 (5.5%), and stroke mimic in 291 (16.0%). Vascular imaging was not performed in 289/1110 (26%) of ischemic stroke patients. Time from symptom onset to EMS alert by patient or relatives and to hospital arrival were 23 min [6–78] and 79 min [53– 138] respectively. Wake-up stroke represented 20.9% of the sample
Supplementary file 1
The RACE scale showed a strong correlation with the NIHSS evaluated at hospital arrival (r=0.74, P<0.001). Equivalences between the RACE and NIHSS scores are shown in Figure S1 (supplemental appendix). Forty-eight percent of the patients scored RACE ≥5. Median RACE score was significantly higher in ischemic stroke patients with LVO (RACE, 7 [5–8]) and intracerebral hemorrhage (6 [4–7]) than in ischemic stroke patients without LVO (3 [2–6]), TIA (2 [1– 4]), and stroke mimic (2 [2–4]) (P<0.001).
Predictive value of the RACE scale for large vessel occlusion
Large vessel occlusion was documented in 309/1533 patients (20.2%). Diagnosis was performed mostly by CTA in 86% of patients, and less frequently (in 8%) by MRA or transcranial duplex (in 6%). Site of occlusion was TICA in 18.1%, MCA M1 in 59.5%, tandem in 18.4%, and basilar artery in 3.9% of the patients.
Receiver operating curve demonstrated that the RACE scale was highly effective in identifying patients with LVO (c-statistic, 0.77: 95% CI 0.75 to 0.80). The cut-off RACE ≥5 showed sensitivity 0.84, specificity 0.60, positive predictive value 0.35, and negative predictive value 0.94 for detecting LVO. One-third of the patients with a RACE ≥5 had LVO (35%), compared with 6% of those with a RACE <5 (P<0.001). The higher the cut-off point of the RACE, the lower the sensitivity and the higher the specificity and positive predictive value to identify LVO (figure 1). Results for other cut-off points are shown in table 1.
Sensitivity (squares), specificity (circles), and positive predictive value (cross) of different cut-off values of the RACE scale for the detection of large vessel occlusion.
Sensitivity specificity, positive predictive value (PPV), and negative predictive value (NPV) of different cut-off values of the RACE scale for the detection of large vessel occlusion
When distal MCA-M2 was included in the definition of LVO, a cut-off point of RACE ≥5 showed lower sensitivity (0.79), which means that 21% of patients with LVO+M2 were missed, but specificity and PPV was higher (0.62 and 0.42).
The RACE scale showed a good accuracy in identifying posterior circulation occlusions, with 11 of 12 patients with basilar occlusion scoring RACE ≥5.
Misclassified patients using the RACE scale
A cut-off point of RACE ≥5 correctly classified a total of 997/1533 (65%) patients regarding the presence or absence of LVO. There were 488 (32%) false positive (scoring RACE ≥5 and not having LVO) and 48 (3%) false negative patients (having LVO and scoring RACE <5) (figure 2).
Distribution of ischemic stroke with large vessel occlusion (LVO; black), without LVO (gray), intracerebral hemorrhage (dashed), and stroke mimics (white) across RACE scores.
False positive patients were mostly intracerebral hemorrhage (29% of those with a RACE ≥5), and less frequently ischemic stroke with no LVO (25%, including 7% with MCA-M2 occlusion), stroke mimic (8%), or TIA (3%). A low proportion of patients with a RACE ≥5 were diagnosed of ischemic stroke patients with no LVO requiring iv tPA (9.5%).
False negative patients were principally patients with LVO with mild or moderate symptoms severity. Compared with true positive patients, false negative patients had lower stroke severity (median RACE 3 [2–4] and median NIHSS 13 [8– 18] vs. RACE 7 [6– 8] and NIHSS 19 [16– 22], P<0.001 for both comparisons) and more frequently scored NIHSS <6 (15% vs 2%, P<0.001). False negatives showed more frequently tandem occlusion (33% vs 16%), less frequently ACM-M1 (52% vs 61%) and TICA (12% vs 19%) occlusions than true positives, with no differences on basilar occlusion (2.1% vs 4.2%).
Predictive value of the RACE scale for endovascular treatment
For this analysis, all 1822 stroke code patients (with or without vascular imaging) were included. A total of 229/1822 patients (12.6%) underwent EVT. Receiver operating curve demonstrated a good predictive capacity of RACE scale in identifying patients who received EVT thereafter (c-statistic, 0.69: 95% CI 0.66 to 0.73). The cut-off RACE ≥5 showed sensitivity 0.75, specificity 0.55, positive predictive value 0.20, and negative predictive value 0.94 for EVT treatment. Endovascularf treatment was given to 20% of patients with a RACE ≥5 compared with 6% of those with a RACE <5. In those directly admitted at CSC, these proportions were 26% and 8% respectively.
Distance to a comprehensive stroke center and access to endovascular treatment in patients with suspected large vessel occlusion
From all those patients with a suspected LVO (RACE ≥5, n=880), 52% were admitted at a CSC and 48% were first admitted at a local center and transferred to the CSC for EVT (drip-and-ship). Baseline characteristics and differences between groups are shown in Table S2 (supplemental material).
The proportion of patients treated with iv-tPA was similar across center groups (25%–30% approximately). In contrast, access to EVT was higher in patients admitted directly to CSCs compared with patients referred from a local center (26% vs 12% of those with LVO suspicion; 62% vs 38% of those patients with confirmed LVO; P<0.001) (Table S2). Direct admission at a CSC was independently associated with receiving EVT in a logistic regression analysis adjusted by age and baseline NIHSS compared with being referred from a local center (OR 2.40; 95% CI 1.66 to 3.46).
Regarding time metrics, time from onset to groin puncture was 146 min longer for patients first presenting to a local center compared with those admitted directly at a CSC. This delay was mainly explained by higher in-hospital times in the referral center (door to neuroimaging and door to iv-tPA), added to the transfer time to the CSC. Overall, time expended at the referral center (door in to door out time) was 95 min (Table S2). Figure 3 shows median times metrics from onset to recanalization in patients finally treated with EVT.
Mean time intervals from stroke onset through revascularization in patients treated with endovacular treatment. Comprehensive stroke centers (CSC): patients directly attended in a CSC. RC: referral center; GP: groin puncture.
Discussion
This large validation study performed after the implementation of the RACE scale in the real clinical practice in the entire region of Catalonia confirms a reasonable accuracy of the RACE scale in identifying LVO patients and candidates for EVT. A RACE score ≥5 detected 75% of the patients who finally underwent EVT, confirming the scale as a valuable tool at a prehospital level. Importantly, by confirming that the results obtained in this province-wide implementation are similar to the pilot study5 we were able to validate the RACE scale as an accurate and reproducible measurement tool. Moreover, this study offers relevant information about geographical inequalities on the access to EVT of patients with suspected LVO and reopens the debate regarding the need to implement improvement measures.
In the last years, numerous prehospital tools aiming to promptly identify EVT candidates have been described.6–9 The optimal characteristics that should have a score to identify endovascular candidates (patients with LVO) have been proposed recently.10
First, the optimal scale must be validated in external data sets and in the prehospital setting, including all the patients with a suspected stroke (ie, ischemic, hemorrhage, TIA, and mimics). Most of the studies have been performed in selected, mainly in-hospital, cohorts. The LAMS11 and CPSSS12 scales have been validated at the prehospital level but only in short cohorts of less than 100 patients. One of the strengths of the RACE scale is that it is the first clinical tool validated prospectively in two different large cohorts after being implemented in a regional stroke code system. Unfortunately, this study does not allow for a direct comparison between different scores applied in the field by EMS.
Second, prehospital scales should be highly accurate to identify LVO. In general, sensitivity to identify LVO is high for all the published scales, but positive predictive value is relatively low, between 35% to 50% accordingly to different studies6–8 yielding a notable number of false positive patients with no LVO erroneously identified as EVT candidates, most of these patients being diagnosed with intracerebral hemorrhage (29% of selected patients accordingly with our results). In the future, prehospital tools should be refined by adding other clinical items to clinical scales or combining these clinical scales with technological diagnostic methods. In the meanwhile, before implementing a triage protocol, it is necessary to decide the preferred equilibrium between sensitivity and PPV, in order to avoid losing LVO patients but minimizing the proportion of false positives. In our opinion, taking into account the devastating outcome of acute stroke due to LVO and the close relation between time to treatment and recovery, it is preferable to prioritize sensitivity over PPV. A RACE scale ≥5 meets this objective, identifying 84% of LVO patients but assuming that 65% of selected patients do not really have a proximal LVO (it is worth highlighting that an extra 10% approximately of selected patients will have ACM-M2 occlusion). The grading system offered by the RACE scale allows for the adjustment of the cut-off point, in either direction, one that would allow identifying more LVO (being more sensible) or one that would yield a higher percentage of LVO (being more specific), as shown in figures 1 and 2. An over triage of patients implies a possible overcrowding of CSC as well as possible harm to particular patients, especially those with no LVO. According to our results, a triage protocol based on a RACE ≥5 would result in 26% of patients receiving EVT, 29% patients with intracerebral hemorrhage, 8% stroke mimic, and 37% ischemic stroke or TIA not receiving EVT. However, one should take into account that, based on previous studies, patients with intracerebral hemorrhage may also benefit from receiving specialized attention at a CSC.13 Importantly, in the same direction of previous data,7 only 9% of selected patients with a RACE ≥5 had an non-LVO ischemic stroke eligible for iv tPA, and for this small subgroup of patients, bypassing the local center could imply a delay in the initiation of iv tPA treatment. Considering the distribution of final diagnostic and treatment options, it is possible that a bypass protocol by increasing access to timely EVT of eligible patients, may benefit a larger number of patients (those with LVO and hemorrhage) than it could theoretically harm.
Another characteristic is that prehospital scales must be simple and rapid. The RACE scale is considered too complex in some reports. In our experience, region-wide implementation of the RACE scale is feasible with appropriate training, and evaluating a patient in real practice takes only 2 min. During the study period 76.6% of the stroke code activations were evaluated with the RACE scale. However, its use has risen progressively and nowadays it is evaluated by EMS in more than 95% of stroke code patients and its use has extended to the emergency departments of several telestroke and no stroke-ready centers of Catalonia as a tool to evaluate stroke severity. Moreover, the RACE is being used in other regions, supporting its feasibility regarding training and implementation.14 A secondary analysis of the pilot study showed that simpler versions of the RACE imply a significantly poorer accuracy of identifying LVO, and thus, it is preferable to use the original version.15
Finally, prehospital tools must be proven to improve patient outcomes. Our study provides novel data on the clinical implications of using a triage tool. Patients with RACE ≥5 first attending a referral center are 2.5 times less likely to receive EVT and are treated 2 hours' later, compared with patients directly attening a CSC. These results reopen the debate about potential solutions to resolve this inequity, such as opening new CSCs in remote areas, streamlining the pre- and in-hospital process,16 or implementing direct transfer protocols of remote population directly to a CSC. As shown in figure 3, for EVT-eligible patients located in remote areas, time from onset to treatment could be reduced by around 70 min if they have followed a bypass protocol directly to a CSC, and the proportion of LVO patients receiving EVT could be closer to the rates achieved in areas covered by CSCs. In the same direction, some non-controlled observational studies compare the clinical outcome of patients treated with EVT referred from other hospitals or directly attendeding the CSC, demonstrating in general a better outcome in favor of direct transfer.17 18 Other studies create mathematical models that recommend direct or drip-and-ship transfer based on clinical and geographical characteristics, showing that the higher the RACE scale, the higher the benefit of a mother ship model.19 20 The present study adds new information about the inequity of the access to EVT in remote areas from a population point of view, considering all patients with the suspicion of LVO, that have lower odds of receiving EVT and a longer time to treatment. Nevertheless, future randomized clinical trials, such as the ongoing RACECAT trial (NCT02795962) performed in Catalonia, will address questions regarding the clinical benefit of direct transfer to a CSC of EVT candidates selected at a prehospital level.
Our study has some limitations. First, the study did not include all the stroke code activations, since 23% of them had no evaluation on the RACE scale. Patients excluded had less severe symptoms and a lower proportion of LVO: therefore presumably some patients with atypical, minor, or vertebra-basilar symptoms with LVO may have been missed. Moreover, since this study is based on usual clinical practice, 26% of ischemic stroke patients were not evaluated with a neurovascular imaging to surely identify LVO, and these patients were excluded from the accuracy analysis. For some patients LVO status was not assessed because of age and comorbidity. However, patients without vascular assessment had also lower NIHSS, suggesting no LVO or spontaneous recanalization, but leaving open the possibility that some minor stroke with LVO may also have been missed. Inclusion of the whole sample could slightly modify the final results. Regarding the final treatment offered to patients, the CICAT registry includes the type of reperfusion treatment but does not detail the particular reason for excluding patients for EVT. Although some few cases transferred from a local center may arrive at the CSC with complete recanalization (a condition that occurs in less than 20% of LVO patients after iv tPA), we presume that most of the transferred cases are excluded because they already had an established infarct with no salvageable brain tissue. In this sense, we think that there is room for improvement in the rate of patients treatable with EVT, that falls from 62% of all LVO patients in areas covered by CSC to 38% in remote areas. Finally, analysis of the impact of the RACE score and the transfer protocol on the clinical outcome was not possible since 50% of the sample had no clinical follow-up.
In conclusion, this study confirms the accuracy of the RACE scale to identify patients with LVO that may benefit from EVT with a high sensitivity and moderate specificity. A RACE score ≥5 detected 75% of the patients who finally underwent EVT confirming the scale as a valuable tool at the prehospital level. Access to EVT of patients triaged with a RACE ≥5 located in remote areas is half that for patients covered by a CSC. Thus, establishing direct transfer protocols to a CSC could increase the rate of EVT and the odds of clinical recovery in patients located in remote areas.
Acknowledgments
This work has been performed on behalf of the Catalan Stroke Code and Reperfusion Consortium (Cat-SCR). We thank Dr. Tudor Jovin for the edited corrections of this manuscript.
References
Footnotes
Contributors MR, MM, AR_C, PC, XU, AR-C, LP, FP, JS, DC, JZ, JAK, XU, JS, and SG and other members of the Catalan Stroke Code and Reperfusion Consortium (Cat-SCR) got patients' data and revised the paper. MG, MQ, MAM, and XJ coordinated EMS protocols and registry of data. DC, NPO, MG, MS, MAM, and XJ implemented the tool studied in this trial. SA, MR, NPO, PC, XU, and AD designed data collection tools. SA assured the quality of the prospective data registry and coordinated data extraction. DC and NPO designed the study, cleaned, and analyzed the data, and drafted and revised the paper. and, writing of the manuscript. AD supervized and revised the whole project.
Funding This work was supported by the Spanish Ministry of Health, Instituto de Salud Carlos III [name of funder] grant number FIS PI 16/01849.
Competing interests Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Correction notice Since this paper was first published online, data in the abstract has been updated.
Patient consent for publication Not required.