Emergency medical services/original research
Out-of-Hospital Stroke Screen Accuracy in a State With an Emergency Medical Services Protocol for Routing Patients to Acute Stroke Centers

Presented at the International Stroke Conference, February 2013, Honolulu, HI.
https://doi.org/10.1016/j.annemergmed.2014.03.024Get rights and content

Study objective

Emergency medical services (EMS) protocols, which route patients with suspected stroke to stroke centers, rely on the use of accurate stroke screening criteria. Our goal is to conduct a statewide EMS agency evaluation of the accuracies of the Cincinnati Prehospital Stroke Scale (CPSS) and the Los Angeles Prehospital Stroke Screen (LAPSS) for identifying acute stroke patients.

Methods

We conducted a retrospective study in North Carolina by linking a statewide EMS database to a hospital database, using validated deterministic matching. We compared EMS CPSS or LAPSS results (positive or negative) to the emergency department diagnosis International Classification of Diseases, Ninth Revision codes. We calculated sensitivity, specificity, and positive and negative likelihood ratios for the EMS diagnosis of stroke, using each screening tool.

Results

We included 1,217 CPSS patients and 1,225 LAPSS patients evaluated by 117 EMS agencies from 94 North Carolina counties. Most EMS agencies contributing data had high annual patient volumes and were governmental agencies with nonvolunteer, emergency medical technician–paramedic service level providers. The CPSS had a sensitivity of 80% (95% confidence interval [CI] 77% to 83%) versus 74% (95% CI 71% to 77%) for the LAPSS. Each had a specificity of 48% (CPSS 95% CI 44% to 52%; LAPSS 95% CI 43% to 53%).

Conclusion

The CPSS and LAPSS had similar test characteristics, with each having only limited specificity. Development of stroke screening scales that optimize both sensitivity and specificity is required if these are to be used to determine transport diversion to acute stroke centers.

Introduction

Reperfusion therapy and other advances in stroke therapy during the past 2 decades highlight the critical role of emergency medical services (EMS) in optimizing acute stroke care.1, 2, 3, 4, 5 Most important, early administration of intravenous tissue plasminogen activator in selected patients with acute ischemic stroke increases the likelihood of a favorable outcome, especially when administered within 90 minutes of symptom onset.3, 6, 7, 8 EMS protocols routing patients with acute stroke to hospitals capable of delivering intravenous tissue plasminogen activator is one strategy to improve timely use of thrombolytic therapy.1, 9, 10, 11, 12, 13 Such protocols rely on the use of sensitive and specific stroke screening scales to identify patients most likely to benefit from transport to a stroke center. The most commonly used stroke screening instruments are the Cincinnati Prehospital Stroke Scale (CPSS) and the Los Angeles Prehospital Stroke Screen (LAPSS).1, 14, 15 Although the CPSS and the LAPSS are widely promoted and broadly adopted, the generalizability and overall accuracy of the LAPSS and CPSS are unclear.1, 13, 16, 17, 18, 19, 20, 21

Many states, including North Carolina, have implemented statewide stroke patient EMS routing plans that specify that, within certain time constraints, patients with a positive stroke screen be transported to hospitals designated as acute stroke centers.11 Determining the accuracy of out-of-hospital stroke screens in such states is essential because the use of low-sensitivity screens can result in transport to hospitals unable to treat patients with thrombolytics, whereas bypassing hospitals according to false-positive screens can result in costly and inconvenient transport diversion. Among the challenges of studying stroke screen accuracy is the ability to link out-of-hospital databases containing EMS stroke screen results with hospital databases containing patient diagnostic data. PreMIS (Prehospital Medical Information System) is a statewide EMS database used throughout North Carolina.22 It includes patient identifiers and data fields for CPSS and LAPSS results. The North Carolina Disease Event Tracking and Epidemiologic Collection Tool (NC DETECT) is a deidentified statewide hospital emergency department (ED) surveillance database that includes ED diagnoses.23 Access to both of these out-of-hospital and hospital databases is unique to North Carolina. We have been successful in using deterministic matching to link PreMIS to deidentified databases such as NC DETECT.24

The purpose of this study was to conduct a statewide assessment of the accuracy of the CPSS and LAPSS in identifying stroke patients by comparing the stroke screen results in PreMIS with the ED diagnostic information contained in NC DETECT.

Section snippets

Study Design and Setting

As a substudy of a project evaluating North Carolina’s EMS routing protocol for patients with suspected stroke, we conducted a retrospective study of patients from January 1, 2009, to March 31, 2011.

Data Collection and Processing

We used PreMIS, which is a National Emergency Medical Services Information System–compliant, Internet-based system for documenting EMS service delivery and care for patients in North Carolina.22 EMS agencies collect and submit data into PreMIS by using either a Web-based interface provided at no

Characteristics of Study Subjects

The Figure provides information on the study sample. We were able to link 62% of PreMIS records to NC DETECT records. Our final study sample included CPSS or LAPSS data from 2,442 patients, generated by 117 EMS agencies from 94 of North Carolina’s 100 counties. It included 1,217 patients with CPSS data and 1,225 patients with LAPSS data. Characteristics for patients screened with the CPSS versus the LAPSS are listed in Table 1. For both stroke screens, most EMS agencies contributing data had

Limitations

Our study has several limitations. Our final study sample was limited by missing ICD-9 codes for some patients and the overall linkage rate. However, according to our previously published analysis, we are unable to postulate systematic bias involving linked versus nonlinked records that should affect our stroke screen analysis. Although we found both the CPSS and LAPSS to be reasonably sensitive, our methodology favored overestimating sensitivity, according to the selection criterion of cases

Discussion

We evaluated the accuracy of the CPSS and the LAPSS in the setting of a statewide EMS system. Although the sensitivities and specificities of the 2 scales were similar, the specificities were at best modest. Because poor specificity can result in “overtriage,” with many nonstroke patients being diverted to stroke centers, specificity assumes particular importance when stroke screens are used for transport diversion. Accepting the precept that some level of overtriage is justified to improve

References (32)

  • J.L. Saver et al.

    Time to treatment with intravenous tissue plasminogen activator and outcome from acute ischemic stroke

    JAMA

    (2013)
  • R. Higashida et al.

    Interactions within stroke systems of care: a policy statement from the American Heart Association/American Stroke Association

    Stroke

    (2013)
  • J.E. Acker et al.

    Implementation strategies for emergency medical services within stroke systems of care: a policy statement from the American Heart Association/American Stroke Association Expert Panel on Emergency Medical Services Systems and the Stroke Council

    Stroke

    (2007)
  • S. Song et al.

    Growth of regional acute stroke systems of care in the United States in the first decade of the 21st century

    Stroke

    (2012)
  • R. El Khoury et al.

    Overview of key factors in improving access to acute stroke care

    Neurology

    (2012)
  • C.S. Kidwell et al.

    Identifying stroke in the field. Prospective validation of the Los Angeles Prehospital Stroke Screen (LAPSS)

    Stroke

    (2000)
  • Cited by (40)

    • European Resuscitation Council Guidelines 2021: First aid

      2021, Resuscitation
      Citation Excerpt :

      In many of the prehospital setting studies, the stroke assessments were performed by paramedics or nurses4,5,51 so this guideline was based on extrapolation of potential benefit when these tools are used by lay people or first aid providers. The specificity of stroke recognition can be improved by using a stroke assessment tool that includes blood glucose measurement such as the LAPSS52–56 or MASS.53,54,57 (weak recommendation, low certainty evidence). However, it is recognised that not all first aid providers have access to or the skills or the authority to use a calibrated glucose measurement device.

    • 2020 International Consensus on First Aid Science With Treatment Recommendations

      2020, Resuscitation
      Citation Excerpt :

      Of patients who had the scale applied, 19.1% received thrombolytic therapy compared with 7.5% who did not have the scale applied (RR, 2.56; 95% CI, 1.02–6.45). For the important outcome of recognition of stroke (diagnostic studies, outcome defined as correct stroke diagnosis), we identified 19 observational studies62–65,68–71,73–82,84 including 8153 participants, studying 9 different screening tools (FAST, LAPSS, OPSS, CPSS, ROSIER, MASS, BEFAST, MedPACS, PreHAST). All studies used the same positivity threshold for each scale (1 or greater).

    • Emergency medical services for acute ischemic stroke: Hub-and-spoke model versus exclusive care in comprehensive centers

      2019, Journal of Clinical Neuroscience
      Citation Excerpt :

      This approach puts the burden on EMS to recognize signs of LVO, through published pre-hospital grading scales [7–9]. However, this proposition has been met with significant resistance from physicians, pointing out the significant risk of misclassification [10–14]. Of all patients presenting with neurologic compromise only a small minority suffers from an LVO, while available pre-hospital grading scales have only moderate sensitivity and specificity, even in controlled settings.

    • Derivation and Validation of the Emergency Medical Stroke Assessment and Comparison of Large Vessel Occlusion Scales

      2018, Journal of Stroke and Cerebrovascular Diseases
      Citation Excerpt :

      Prehospital stroke screens, including the Cincinnati Prehospital Stroke Scale and the Los Angeles Prehospital Stroke Screen, were developed to help distinguish stroke from stroke mimic.2-6 Unfortunately, first-generation prehospital stroke screens have had limited accuracy when they have been tested in the field.7-10 Recently completed randomized clinical trials have demonstrated the efficacy of endovascular therapy (ET) with stent retrievers versus IV t-PA alone in patients with large vessel occlusion (LVO) and salvageable tissue, generally in patients presenting with a National Institutes of Health Stroke Scale (NIHSS) greater than or equal to 6 with groin puncture less than 6 hours of ischemic stroke onset.11-16

    • Recent Endovascular Stroke Trials and Their Impact on Stroke Systems of Care

      2016, Journal of the American College of Cardiology
    View all citing articles on Scopus

    Please see page 510 for the Editor’s Capsule Summary of this article.

    Supervising editor: Donald M. Yealy, MD

    Author contributions: AWA conceived the study and obtained research funding. SW managed the data. SW, WDR, and JS provided statistical advice. SW analyzed the data. AWA drafted the article, and all authors contributed to its revision. AWA takes responsibility for the paper as a whole.

    Funding and support: By Annals policy, all authors are required to disclose any and all commercial, financial, and other relationships in any way related to the subject of this article as per ICMJE conflict of interest guidelines (see www.icmje.org). The authors have stated that no such relationships exist and provided the following details: This work was funded by the 2011-2012 EMF/Genentech Regionalization and Stroke Care Grant.

    The NCDETECT Data Oversight Committee does not take responsibility for the scientific validity or accuracy of methodology, results, statistical analyses, or conclusions presented.

    A feedback survey is available with each research article published on the Web at www.annemergmed.com.

    A podcast for this article is available at www.annemergmed.com.

    View full text