Article Text
PDF
Abstract
Introduction The Speech Arm Vision Eyes (SAVE) scale, a 4-item clinical scale emphasizing binary scoring and avoidance of nuanced examination distinctions, predicts LVOs with similar characteristics as more complex scales.
Methods Receiver operating characteristic analyses of the prospective STOPStroke study assessed the ability of the SAVE scale and other published scales to predict LVO. We identified scale thresholds with positive likelihood ratios with 95% confidence intervals of ≥5.0 or negative likelihood ratios with 95% confidence intervals of ≤0.5.
Results 735patients were studied. LVO prevalence was 33%. Area under the curve was 0.79 for SAVE, 0.82 for FAST-ED, 0.80 for mNIHSS and NIHSS, and lower for all other scales. SAVE=4, EMSA=6, mNIHSS≥10, NIHSS≥16, and RACE≥8 had positive likelihood ratios with 95% confidence intervals ≥5.0. SAVE≥2, CPSS≥2, C-STAT≥1, EMSA≥4, FAST-ED≥3, G-FAST≥3, mNIHSS≥6, NIHSS≥9, PASS≥1, RACE≥2, VAN=1, and 3I-SS≥1 had negative likelihood ratios with 95% confidence intervals ≤0.5.
Conclusions SAVE=4 performed similarly to more complex scales at predicting LVO. Other simplified scales did not have thresholds with positive likelihood ratios with 95% confidence intervals ≥5.0. Validation is need in a prehospital cohort of patients with suspected stroke.
- stroke
- thrombectomy
- artery
Statistics from Altmetric.com
Introduction
Shorter door-to-groin puncture times are associated with better outcomes when using endovascular therapy (EVT) to treat acute ischemic stroke due to a large vessel occlusion (LVO).1 Earlier prediction of whether or not a patient has a LVO could assist with time-sensitive prehospital and emergency department (ED) decisions aimed at decreasing EVT treatment times.2 LVO prediction scales use clinical information to attempt to make this distinction before vessel imaging.
Many scales for predicting LVO have been retrospectively derived from National Institutes of Health Stroke Scale (NIHSS) examination items.3–8 However, none are considered to be more accurate than the others.2 There remains a need for a scale that is short, easy to score, and captures cortical or hemispheric deficits that are highly predictive of LVO stroke. Heldner et al calculated the ability of each item of the NIHSS to predict the presence of an LVO and found that the items with the greatest predictive value were Best Gaze, Motor Arms, Best Language or Extinction, and Visual.9 Other studies have similarly found Best Gaze and Motor Arms to be the first and second most predictive items.4 8 10 11
We created the four-point Speech Arm Vision Eyes (SAVE) scale to combine these highly predictive examination findings while avoiding nuanced assessments for aphasia and neglect. We also limited scoring to binary (absent or present) assessments to avoid the added time and complexity that may come with gradations of severity. We hypothesized that SAVE would predict LVO as well as more complicated scales despite its avoidance of nuanced assessments and straightforward scoring. To test our hypothesis we conducted a retrospective analysis of the Screening Technology and Outcomes Project in Stroke (STOPStroke) Study. The STOPStroke Study was a prospective cohort analysis of consecutive patients who presented to two university hospitals between 2003 and 2006 with symptoms of possible acute ischemic stroke or transient ischemic attack within 24 hours of symptom onset.12
Methods
All patients in the STOPStroke Study were assessed with a NIHSS score, computed tomographic angiography (CTA), and assigned a final diagnosis. NIHSS scores were collected by personnel certified in the scale, which typically includes neurologists and emergency physicians. The STOPStroke Study excluded patients with intracranial hemorrhage, incomplete occlusions, occlusions in multiple vascular territories, or allergies to iodinated contrast.12 A retrospective review of initial NIHSS data determined whether each patient had scored points for any of the four SAVE scale items shown in table 1. The SAVE scale assigns one point for any NIHSS abnormalities of Speech (Best Language and/or Dysarthria), asymmetric Arm weakness (a difference between the two Motor Arm scores), Visual field deficits, or Eye deviation (Best Gaze).
The SAVE scale
LVO was defined as a symptomatic occlusion of the extracranial internal carotid artery (ExICA), intracranial internal carotid artery (ICA), first division of the middle cerebral artery (M1), second division of the middle cerebral artery (M2), or basilar artery (BA).
The ability of the SAVE scale to predict the presence of a LVO on CTA was assessed by receiver operating characteristic (ROC) analyses. Sensitivities, specificities, positive likelihood ratios, negative likelihood ratios, positive predictive values, and negative predictive values were calculated for each scale score threshold along with 95% confidence intervals. The same was done for the Cincinnati Prehospital Stroke Scale (CPSS), Cincinnati Stroke Triage Assessment Tool (C-STAT), Emergency Medical Stroke Assessment (EMSA), Field Assessment Stroke Triage for Emergency Destination (FAST-ED), Gaze-Face-Arm-Speech-Time (G-FAST), the modified NIHSS (mNIHSS), NIHSS, Prehospital Acute Stroke Severity (PASS), Rapid Arterial Occlusion Evaluation (RACE), Vision Aphasia Neglect (VAN), and 3-Item Stroke Scale (3I-SS).3–9 11 13–16 Scales were compared based on the areas under the ROC curve (AUC) and the ability of each scale to provide a threshold where the positive likelihood ratio (PLR) had a 95% CI with a lower limit of ≥5.0 or the negative likelihood ratio (NLR) had a 95% CI with an upper limit of ≤0.5. Analyses were done using STATA-15 software (College Station, Texas, USA).
Results
Of the 735 patients with complete NIHSS and LVO data, 578 patients had acute ischemic stroke, 97 patients had a transient ischemic attack, and 60 patients had stroke mimics. Two hundred and forty-one patients (33%) had LVOs of the ExICA, ICA, M1, M2, or BA with AUCs of 0.79 for SAVE, 0.82 for FAST-ED, 0.80 for mNIHSS, and 0.80 for NIHSS. All other scales had lower AUCs with a range of 0.75–0.78. The SE for each area under the ROC curve was 0.02. A figure demonstrating the close overlap of the ROC curves and exact AUCs for each scale is provided in the online supplementary material. Table 2 provides the diagnostic test parameters of each threshold of the SAVE scale. As the SAVE score threshold increases the sensitivity decreases, the specificity increases, and a smaller percent of patients are predicted to have LVOs.
Supplemental material
Diagnostic test parameters of each SAVE scale threshold
Table 3 provides a comparison of the LVO prediction scales tested in this study by selecting thresholds with high PLRs or low NLRs. Given the wide 95% CIs of many of the scales, the lower limit of the PLR 95% CI was used to strengthen the inference that a positive result increased the pre-test odds of LVO by at least five times. Similarly, we required the upper limit of the NLR 95% CI to be ≤0.5 to ensure a negative test reduced the pre-test odds of LVO by at least half. Diagnostic test parameters for all scales tested at each threshold are provided in the online supplementary material. Online supplementary table 1 includes commonly reported scale thresholds for each scale that did not meet the above criteria for table 3.
LVO prediction scale thresholds selected for high positive likelihood ratios and low negative likelihood ratios.
Discussion
SAVE performed similarly to, or better than, most of the other LVO prediction scales tested in this cohort when assessed by its AUC and its ability to provide a score threshold where the 95% CI of the PLR was ≥5.0. FAST-ED, NIHSS, and mNIHSS had slightly higher AUCs but are significantly more complex. FAST-ED is complex due to its severity gradations, length, and assessment for neglect. Of note, it was derived from the same STOPStroke population as in our study. However our study included extracranial internal carotid LVOs and provided 95% CIs not included in the original FAST-ED study.7 The NIHSS remains a key feature of acute stroke assessments, but it is considered too long and complex for prehospital use.2 It is not yet clear whether this also applies to the mNIHSS, though this scale is longer than most dedicated LVO prediction scales. While it would be logistically preferable to use other existing short and simple scales with binary scoring, such as CPSS, G-FAST, PASS, and VAN, none of these had score thresholds in this study where the 95% CIs of the PLRs were ≥5.0. SAVE=4 meets all these criteria. Each scale was able to provide a threshold with a NLR that had a 95% CI <0.5; however, none of the scales had NLRs with 95% CIs below the more clinically desirable likelihood ratio of 0.2. No scale simultaneously achieved a PLR≥5.0 and a NLR≤0.5 at a single threshold. This limitation might be overcome if physiology-based LVO prediction devices could provide favorable NLRs that could be incorporated into device and examination-based decision rules.17
RACE, C-STAT, and the Los Angeles Motor Scale (LAMS) have been prospectively validated in prehospital cohorts of patients with suspected stroke and warrant special discussion.6 14 18 19 Of these, the RACE scale is the most complex. It requires nuanced assessments of either aphasia or neglect that depend on the side of the weakness.6 Our data suggest that SAVE can provide similarly strong PLRs with less complexity and a slightly higher AUC. C-STAT is also short, uses binary scoring, and tests arm weakness and gaze deviation. It differs from SAVE by asking the patient to say their name, age, and to follow commands.14 In this study C-STAT did not provide a threshold with a PLR where the 95% CI was ≥5.0. LAMS requires severity gradations of face, arm, and grip motor deficits.20 It was not included in this analysis because grip strength data were not available. SAVE will need to be prospectively validated in a prehospital cohort before more meaningful comparisons with these scales can be made.
This study has important strengths. First, the inclusion of patients with suspected stroke, rather than only those with confirmed strokes, allowed for the scales to be tested with a more appropriate cohort with 60 stroke mimic patients. Second, the inclusion of patients within the 24 hours timeframe increases its generalizability to modern stroke triage decisions. Third, our results are more readily translatable to clinical decision-making owing to the emphasis on likelihood ratios and confidence intervals rather than sensitivities and specificities. Likelihood ratios are predictive measures that convey, in this study, the extent to which the pre-test probability of LVO is increased by a positive test or decreased by a negative test.21 Likelihood ratios of 5–10 and 0.1–0.2 are considered to result in a moderate change between pre-test and post-test probability. In contrast, likelihood ratios of 2–5 and 0.5–1 are considered to result in small changes.22 For example, owing to the need for conversions between odds and probabilities, a likelihood ratio of 5 increases the probability of disease by approximately 30% while a likelihood ratio of 0.5 decreases the probability of disease by approximately 15%.21 Diagnostic test studies without 95% CIs do not give the reader a sense of the precision of the estimates in that study. An impressive likelihood ratio with a wide but unreported CI could be unintentionally misleading.23 24 Finally, our online supplementary materials provide detailed diagnostic test accuracy results at various thresholds, including likelihood ratios and confidence intervals, for all the tests studied.
We believe that the SAVE scale could be readily adopted by emergency medical services providers for several reasons. First, the SAVE scale is short and intentionally avoids nuances such as aphasia or neglect testing that would require long, complex, and difficult to sustain training efforts. Its binary scoring of each item is intended to reduce the need for subjective distinctions. This is also facilitated by collapsing aphasia and dysarthria into ’Speech' as has been done in the widely adopted CPSS.13 This also holds true for the assessment of asymmetric arm weakness. McMullan et al showed that C-STAT could be implemented without a formal emergency medical services training programme and gathered feedback that gaze deviation testing could benefit from instructions clarifying what to do when patients’ eyes are closed.14 As a result, the simplified instructions that accompany the SAVE scale in table 1 provide an explanation in plain language as to how this should be approached. A reasonable concern about the SAVE scale is that, to the best of our knowledge, there have not yet been studies reporting the use of visual field testing as part of prehospital stroke assessments. However, visual fields were tested in the ED-based VAN study and are routinely used as part of the NIHSS.3 Visual field testing is worthwhile because of its favorable inter-rater reliability as part of the NIHSS, its neuroanatomical coverage of the temporal, parietal, and occipital lobes contralateral to the side tested, and its use in clinical practice to help distinguish encephalopathic or unresponsive patients with diffuse impairments from those with focal diseases such as LVO.16 It is also abnormal in both anterior and posterior circulation LVOs. We have attempted to create plain language instructions to facilitate its use in table 1.
The usefulness of SAVE, or any other LVO prediction method, inextricably depends on the pre-test probability (prevalence) of LVO. The prevalence in prehospital patients with suspected stroke is unknown and will probably vary geographically, by the method used to classify patients as suspected stroke, and by the definitions of LVO used in each study. Estimates range from 5% to 48% in studies with different approaches to patient selection.19 25 The American Heart Association/American Stroke Association dealt with this uncertainty in their 2018 LVO prediction instruments meta-analysis by demonstrating the range of post-test probabilities that could occur with LVO prediction scales over a range of pre-test probabilities (LVO prevalences) of 5–20%.2 The LVO prevalence of 33% in our study is higher than would be expected due to the exclusion of intracranial hemorrhage and represents an ED, rather than prehospital, population with suspected stroke. However, the same problems apply. As shown in figure 1, if the prevalence of LVO was 10% rather than 33%, the PLR of >5 that comes with a SAVE=4 scale would lead to a post-test probability that barely exceeds 50%.
Post-test probabilities of LVO prediction scales are dependent on the prevalence of LVO. The top graph shows a prevalence of 33% and the bottom graph a prevalence of 10% with resulting post-test probabilities after positive and negative results. Color outlines provide an estimate of 95% confidence intervals. LVO, large vessel occlusion; SAVE, Speech Arm Vision Eyes. (Adapted with permission from Michael Power, Sara Graziadio and Joy Allen. A ShinyApp tool to explore dependence of rule-in and rule-out decisions on prevalence, sensitivity, specificity, and confidence intervals. NIHR Diagnostic Evidence Co-operative Newcastle. July 2017. https://micncltools.shinyapps.io/ClinicalAccuracyAndUtility/.)
For prehospital triage, the range of LVO prevalences for which a test meaningfully changes decisions is bound by the performance of the test, the cost of the test, the costs of diverting patients without LVO to EVT centers (false positives), and the cost of failing to identify and divert a patient with LVO (false negatives).23 Several scales perform similarly, are free to use, and in the case of shorter and less complex scales would not take much time. More importantly, we do not know the cost of diverting patients without LVO (false positives) but expect it to vary according to regional factors, such as the distance to the nearest EVT capable center. Concerns that diversion to an EVT capable center would lead to missed thrombolysis opportunities have been partially alleviated by thrombectomy registry based simulation data, which suggest that, if limited to a 20 mile radius, thrombolysis may be delayed by 7 min while thrombectomy times could be hastened by 94 min.26 The cost of failing to divert a patient with LVO (false negatives) is probably very high given the time-sensitive nature of treatment in the early window, the association with worse outcomes when transfer is required, and profoundly higher rates of disability for patients who are not treated.1 26–28 These costs are worth further quantifying in future studies. If the cost of failing to divert a patient with LVO to an EVT capable center is overwhelming, the optimal strategy in some regions may be to divert all patients with suspected stroke to EVT capable centers. Until we have data to inform these decisions, LVO prediction scales provide a rational means to deal with concerns that diverting all patients with suspected stroke to EVT capable centers will overwhelm their systems.
The situation dramatically changes if LVO prediction scales are used to reach a different decision. For patients who have already arrived in the ED of an EVT capable center, a LVO prediction scale using a threshold with a high PLR could be used to design a system to select patients very likely to have a LVO stroke to hasten EVT door-to-arterial puncture times. For example, if a patient presents directly to an EVT capable center ED and is found to have a SAVE scale score of 4 during the initial assessment, they could be sent directly to a neuroangiography suite equipped to perform a non-contrast head CT and CTA. If appropriate, the patient could then be given intravenous thrombolysis in the neuroangiography suite as the team proceeds to CTA and prepares for conventional angiography. With a SAVE scale score of 4, the combination of the prevalence of LVO in this ED population and a PLR >5 suggests that that the patient would be very likely to have a LVO and unlikely to be a false positive. The cost of failing to identify patients with a LVO already in the ED of an EVT center (false negatives) would be low because these patients would proceed along a standard stroke protocol that performs a CTA in the ED before considering transportation to a neuroangiography suite. Potential costs of triaging patients without LVO to this EVT fast-track (false positives) would comprise practical considerations such as false thrombectomy team activations and risks associated with leaving the ED with a non-LVO diagnosis. Selecting a test with a high PLR would minimize these costs, and future stroke treatment rooms would ideally be in or near hospital EDs.
Our study has a number of weaknesses. First, the exclusion of patients with intracranial hemorrhage from the STOPStroke cohort limits its generalizability to a true patient cohort with suspected stroke. Second, prehospital LVO prediction scales are ideally prospectively validated in a prehospital cohort of patients with suspected stroke.2 Third, in addition to being unable to include LAMS in this analysis we were similarly unable to study the Japan Urgent Stroke Triage (JUST) score because it requires data on symptoms such as headache, dizziness, and nausea, which were not available.29 We were also unable to study the Ambulance Clinical Triage for acute stroke treatment (ACT-FAST) algorithm because it requires patient thrombectomy eligibility data that were not available.30
Conclusion
The SAVE scale uses the presence or absence of speech abnormalities, asymmetric arm weakness, visual field deficits, and gaze abnormalities to predict the presence of a LVO in patients with suspected stroke. It is short, easy to score, and assesses cortical or hemispheric deficits while avoiding nuanced assessments of aphasia and neglect. A score of 4 is highly predictive of LVO with performance similar to more complex scales and better than other short and simplified scales in this study. Region- and decision-specific analyses will depend on the prevalence of LVO and costs of false positives and negatives. In addition to further testing in the prehospital setting, prospective validation as a tool to identify emergency department patients ideal for a ‘direct to neuroangiography suite’ LVO fast-track protocol is another area of interest for future research.
Acknowledgments
Dr. Kevin J Keenan’s research effort during the final drafting ofthis manuscript was supported by the National Institutes of Health StrokeNetFellowship through grants U10NS086494 and U24NS107229.
References
Footnotes
Contributors KJK contributed to the literature search, study design, data analysis, data interpretation, table and figure design, and manuscript writing. WSS contributed to the original data collection, study design, data interpretation, table and figure design, and manuscript writing.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient consent Not required.
Ethics approval University of California, San Francisco Institutional Review Board.
Provenance and peer review Not commissioned; externally peer reviewed.