Introduction: ischemic stroke and artificial intelligence

Acute ischemic strokes caused by large vessel occlusions (LVOs) require emergent detection in pre-hospital triage, similar to the analogous detection of acute myocardial infarcts by electrocardiograms.1 The subsequent in-hospital verification of LVOs and treatment by endovascular thrombectomy (ET) are critical for reducing morbidity and mortality.2–4

The number needed to treat to reduce disability by at least one level on the modified Rankin Scale for one patient with an LVO is 2.0–2.6.2 4 LVOs are detected from ASPECTS (Alberta Stroke Programme Early CT Score) or non-invasive cerebral angiography and perfusion studies that also measure infarct core and penumbral volume mismatch. The results of these studies inform candidacy for acute ET.5 6

However, clinically salient interpretation of perfusion, angiographic, and ASPECTS is subject to inconsistent local expertise, time delays, and varies between institutions. Furthermore, activation of interhospital communication for LVO triage and transport to a thrombectomy center can be operationally challenging. There is frequent time delay, of nearly 100 min, even in experienced systems, which may result in loss of neurons, increased morbidity, or even exclusion as a candidate for an ET intervention.7 For every 15 min of time reduction to ET recanalization of an LVO, 34 per 1000 treated patients have an improved disability outcome.8 Despite stroke system-optimization efforts, there remains an unmet need for more immediate and standardized time-sensitive stroke detection and triage.

Artificial intelligence (AI) is proposed as a tool to deal with this need. Recently, AI has been applied to standardize accurate LVO stroke detection. AI technology varies among stroke imaging software platforms and remains mostly proprietary. A systematic review of stroke AI software, including AI methodology and validation, has not yet been performed. Before doing so in this study, an introduction to AI fundamentals is reviewed.

We focus on a brief review of machine learning (ML) for image processing since diagnostic stroke imaging modalities fall under this category. ML is an area of AI-related research that provides tools to develop or discover decision-making rules from data. Algorithms used for LVO stroke detection use pure ML on highly specific data and fall into the category of ‘narrow AI’ that performs well only on a well-circumscribed task. In the case of stroke, the primary goal of any AI or ML algorithm is to reliably identify the presence or absence of an LVO from three-dimensional tomographic images.

However, many different approaches to achieving this goal using ML methods can be envisioned. These differ in the choice of input abstraction (ie, image input manipulation, either as manually engineered image abstract feature representations or non-abstract: pixel intensities), function approximation (ie, an algorithmic method of estimating or fitting the value of different image data states that have similar features; this includes methods such as a support vector machine, random forest learning (RFL), or deep neural network), final algorithm output format (eg, instance-level classification and regression, or pixel-level localization), and the degree of human supervisory signal during training (eg, no labels, instance-level labels or pixel-level labels with or without uncertainty). Table 1 lists further definitions and the most commonly used setups for the aforementioned design choices.

In the case of ML used for LVO stroke detection, the input abstraction may be a hybrid of methods. Most function approximations commonly applied to the stroke image input are support vector machine, RFL, or deep neural network, using a classification scheme of the input features. To do so, the function approximation receives a supervisory signal, commonly structured supervised, to the ’gold standard' of the human’s radiological scan interpretation—the presence or absence of an LVO stroke.

A full understanding of which imaging platforms use AI, how accurately the AI performs, and how the clinician can use AI in real time to interpret images is needed. This understanding is necessary for informed interpretation of the scans of any individual patient with a stroke that use AI software. Previous studies have described some forms of AI in stroke. ML has been well described in ASPECTS and perfusion to diffusion mismatch,2 9–11 but there are no published studies on ML use in the diagnosis of ischemic stroke by directly identifying LVOs from source images. Although abstracts describing this have been published,12 13 one retrospective study applying AI to vessel densities as an indirect measure of LVO presence,14 and a single retrospective study using AI analysis of patient demographics and clinical examination characteristics to predict LVO,15 to our knowledge there are no studies describing AI for LVO detection from source CT or MR image files.

Our study is a systematic review of the current landscape of AI in ischemic stroke diagnostics by characterizing: (1) the current literature and (2) new ML diagnostic technology for non-contrast CT, CT angiography (CTA), and CT perfusion (CTP) images that has only recently been made available to clinicians, but not yet fully described yet in the literature. The diagnostic utility of different types of ML in patients with a stroke, with respect to acute diagnosis, triage, and prognosis is compared.

Methods

A systematic review was performed in accordance with PRISMA guidelines using PubMed, Medline, and Embase for all peer-reviewed articles in English using predetermined search terms: ‘artificial intelligence’ or ‘machine learning’ or ‘deep learning’ and ‘ischemic stroke’ or ‘large vessel occlusion’. Studies were included if the primary aim was describing acute ischemic LVO diagnosis or prediction of acute outcome (including recanalization, symptomatic intracerebral hemorrhage, or acute in-hospital infarct growth), but not long-term outcomes. Studies that resulted using this search criteria were excluded only if they did not meet inclusion criteria or their primary aim or outcome was any of the following: association with stroke severity as measured by the National Institutes of Health Stroke Scale (NIHSS), or prognostic or effect of treatment study (eg, modified Rankin Scale score at ≥3 months), an AI development-only study (ie, without AI validation), a chronic stroke burden assessment or stroke MRI volume quantification study, or did not apply AI to raw imaging files. Only the past 5 years’ literature, published after January 1, 2014, was included, which is a timeframe that was felt to optimally capture the emerging technologies without redescribing well-known image analysis and AI technology.

Studies were extracted by a single author and reviewed by a second author independently. Unpublished emerging stroke diagnostic and LVO detection technology using both CTA and CTP was identified by worldwide web searches for the same search terms as used in the above databases. This technology assessment was verified and cross-checked by review of FDA 510(k) and European CE mark documents, manufacturer websites, and conference posters. The present review protocol is not a registered protocol. The last date for which the sources were searched was February 28, 2019. Bias in individual studies was assessed but wide variability in design and statistical calculations of the reviewed studies prevented calculation of principal summary measures; further details are contained in the online supplementary Methods.

Results

Systematic review of AI algorithms for acute stroke diagnostics

A total of 20 studies met all inclusion and no exclusion criteria (figure 1). AI use in acute LVO stroke diagnostics and triage falls under three categories: automatic stroke core and penumbra size and mismatch quantification, detection of vascular thrombi or occlusions, and prediction of acute complications (described below and in table 2).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Systematic literature screening and review protocol for studies included and excluded. Inclusion dates were January 1, 2014 to February 28, 2019.

View this table:

• View inline
• View popup

Table 2

Ischemic stroke studies using artificial intelligence (AI) for acute diagnosis, triage, or complication prediction with performance compared with humans (2014–2019). Controls for bias and possible sources of bias for each study are shown in the online supplementary table I

The majority of current studies describe AI-mediated detection of core stroke lesion volume on CT and MRI scans, and some use these features to predict acute stroke growth.16–19 For non-contrast CT analysis specifically, automated ASPECTS from ML used in conjunction with clinical presentation accurately associates NIHSS score and can be used to select patients for ET.20 21 ML algorithms for ASPECTS most commonly employ RFL. The 10 studies reviewed that use RFL show that the AI often out-performs single radiologist ASPECTS and is non-inferior or even better than consensus ASPECTS. The reported sensitivity of AI algorithm ASPECTS range from 45% to 98%, mean 68%, and specificity ranges from 57% to 95%, mean 81%.9 10 14 16 17 22–27 Use of a convolutional neural network (CNN) for a combined asymmetric middle cerebral artery territory hypodensity and dense vessel detection may have higher performance; however, only area under the curve (AUC) metrics are reported (receiver operating characteristic AUC 92–96%).28 Collectively among these studies, there is significant heterogeneity in study data, including cohort size, gold standard comparison, and time to initial head CT acquisition (table 2, online supplementary table I). Several commercial software platforms offer a ML algorithm for ASPECTS within their software: Brainomix e-ASPECTS (Oxford, UK), Siemens Frontier (Erlangen, Germany), iSchemaView ASPECTS (Menlo Park, California, USA), and others that are in development.

No peer-reviewed studies to date have described AI applied to direct LVO detection from CT angiographic studies. However, abstracts describing this have been published.12 29 We note that abstract extraction partially limits the systematic review below for bias, assessment of heterogeneity, characterization of methods, and other desired comparisons.

Many AI algorithms for different types of LVO characterization have been described. These include detection of vessel density asymmetry on CTA, presence of middle cerebral artery dot sign, bioimpedance asymmetry from phase shift spectroscopy, and true intravascular thrombus versus chronic atherosclerotic plaque.14 30–33 These LVO detection studies variably report algorithm performance compared with individual humans, with broad sensitivities of 67–98% and AUC ranging from 85% to 93%. Many use human consensus scoring of the CTA or ASPECTS as the gold standard for comparison. Of the studies reviewed for AI-mediated LVO detection, four clearly describe use of CNNs, and this ML type is the most common AI used. These studies using a CNN to directly detect LVOs from CTA images, report a sensitivity range from 67% to 94%, mean 85%, and specificity from 52% to 94%, mean 91%.12 14 29

AI algorithms are also used for acute prognosis prediction. This may facilitate immediate treatment planning—for example, whether or not to offer IV tissue plasminogen activator or ET due to possible postintervention complications, or near-term outcome prediction, such as risk of intracranial hemorrhage. In general, AI has benefited these prognoses.21 34–36 For example, one study showed that AI led to an absolute increase of 7% accuracy in predicting a symptomatic intracerebral hemorrhage.37

AI software for acute LVO detection

Several AI software platforms offer LVO detection features, not all of which are available in the USA. Each delivers different degrees of automated perfusion, angiographic vessel image analysis, and AI detection of acute stroke features on imaging. The software platforms are: Brainomix e-Stroke Suite (Brainomix, Oxford, UK/Olea Medical, La Cio-tat, France), GE CT Perfusion 4D (Chicago, Illinois, USA), iSchemaView RAPID (Rapid Processing of Perfusion and Diffusion; Menlo Park, California, USA), and Viz.ai LVO and CTP (San Francisco, California, USA).

Each AI stroke diagnostic platform provides CT perfusion images and color-coded maps of stroke core and penumbra. The software also offer different types of LVO detection, having variation in longevity and methodology, and validation in large clinical trials. Below, we review each platform.

Olea Sphere, which partnered Brainomix in the fall of 2018, joined stroke imaging in 2012. In 2015, Brainomix released their AI e-ASPECTS product. e-ASPECTS uses AI to automatically interpret non-contrast CT scans for both a numerical ASPECTS and a volume of hypodensity from a red voxel-wise ‘heat map’. e-ASPECTS highlights and measures non-acute hypodense volume separately from the acute hypodense region. e-ASPECTS is superior or equivalent to human performance (table 2); however, it is not yet available in the USA.

Brainomix e-CTA uses AI to detect the presence or absence of LVOs and received CE mark certification 2018, but is not yet available in the USA. Validation data have also not yet been reported. The software employs a CNN that takes data from the CTA scan, searches for features corresponding to an LVO, and can directly detect an LVO within 1–2 min. It also performs a collateral assessment, similar to RAPID (described below), in which the symmetry of a contrast-filled vessel is compared left to right. However, it differs from RAPID in that asymmetries are quantified both as a red- orange voxel-wise ‘heat map’ showing the regions with a relative lack of vessel density, and a relative percentage, providing a CTA collateral score.38 In 2018, e-ASPECTS and e-CTA were paired with the Olea Sphere product, which provides automated colorimetric CT and MR perfusion maps, computation of mismatch volume and ratio, and DWI maps. Images are automatically pushed to the PACS system (picture archiving and communication system) and distributed via secure email notifications, a web browser interface, and a mobile app (figure 2).

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Brainomix (A) non-contrast head CT, (B) colorimetric analysis and overlaid e-ASPECTS, (C) e-CTA with LVO marked by red circle, and (D) Olea sphere thresholded core in red (<40% CBF threshold) and penumbra in yellow (>6 s Tmax threshold) for a patient with an acute stroke with a left MCA occlusion. Images courtesy of Brainomix. ASPECTS, Alberta Stroke Program Early CT Score; CBF, cerebral blood flow; CTA, CT angiography; LVO, large vessel occlusion; MCA, middle cerebral artery.

iSchemaView RAPID validated perfusion imaging in stroke. RAPID was validated in the DEFUSE 2 study in 2012 and received FDA 510(k) approval in 2013.11 The RAPID software analyzes CT and MRI perfusion studies in <2 min and generates colorimetric perfusion maps for stroke core and stroke penumbra (figure 3). AI is used for perfusion processing and, with MR studies, DWI coregistration, and then a predefined threshold, is used to process a penumbra and core mismatch volumetric ratio. For CT images, RAPID predicts infarct core volume after thrombectomy with an accuracy of 83%, and RAPID MRI stroke core to penumbra mismatch sensitivity is 100% and specificity is 91%.2 18 39 40 These maps are available for viewing on local PACS-enabled computers, on secure email sent to a predefined set of recipients, and on a mobile application. The AI technology was used in the recent large LVO ET trials: EXTEND IA, SWIFT PRIME, CRISP, DEFUSE 2 and 3, and DAWN.2 3 11 40–42

• Download figure
• Open in new tab
• Download powerpoint

Figure 3

RAPID image processing outputs. (A) CT perfusion (CTP) map of a right middle cerebral artery (MCA) syndrome. The CTP stroke core of 1 mL is represented in pink at the top left and to the right is the penumbra of 129 mL. MR perfusion status is shown 24 hours after thrombectomy below for validation. (B) CT angiography image of a patient with a left MCA occlusion (left) and the RAPID CT angiography detection of a >55% reduction in blood vessel density asymmetry, an indirect suggestion of a relevant flow limiting the underlying large vessel occlusion. Images courtesy of iSchemaView. CBF, cerebral blood flow.

Recently, an automated ASPECTS scoring add-on component based on AI was developed by iSchemaView, which is similar to that of Brainomix, but is not yet available in the USA. The AI relies on RFL and creates a mask to compare the affected side with the corresponding brain images in the opposite brain hemisphere. The output is the numerical ASPECTS. CTA data are also provided, and on the mobile device app and RAPID web browser, a rotational view of the source images and maximum intensity projections can be seen. In 2018, a CTA vessel density detection feature was added to identify relative distal MCA vessel asymmetries, which are suggestive of an LVO. However, to date, no published validation are data available for this CTA software. In December 2018, iSchemaView received 510(k) approval for a thrombectomy selection guide. This incorporates inclusion criteria from major clinical trials that used RAPID for patient selection and applies this to the individual scan, providing a simple binary output of whether or not the patient is a candidate for thrombectomy. Finally, if ET is selected for the individual patient, then the interpreting provider manually activates all stroke treatment systems.

GE Healthcare also offers a platform for perfusion imaging in patients with suspected LVO. The software, FastStroke and CT perfusion 4D Neuro, is, similarly, automated software for analyzing CT angiographic and perfusion images related to stroke and tumor angiogenesis. It offers color-coded arterial and venous displays termed ColorViz for collateral circulation assessment and unique volumetric visualization of functional core and penumbra perfusion maps. Further software details and AI algorithm metrics were not provided by GE.

The newest AI stroke diagnostic software is from Viz.ai. It received de novo regulatory clearance from the FDA in February 2018 for the first computer-aided triage and notification platform to identify LVO strokes in CTA imaging (Viz LVO). This was followed in April 2018 with 510(k) clearance for Viz.ai’s CTP software. The Viz LVO and Viz CTP software platforms encompass similar perfusion features to the CTP outputs by iSchemaView, Brainomix/Olea Sphere, and GE 4D Neuro, but combine additional components for automating expedited stroke care—namely, (1) delivery of dynamic CTP and CTA images directly to the users’ mobile device for real-time 3D image manipulation, (2) automatic direct LVO detection from CTA data (Viz LVO), (3) in-built motion correction capability within Viz CTP, and (4) automatic activation of emergent treatment systems when a LVO is detected (figure 4). In comparison, for European users only, Brainomix is also beginning to offer a mobile device PACs feature platform and direct LVO detection.

• Download figure
• Open in new tab
• Download powerpoint

Figure 4

Viz.ai stroke triage software. From left to right, the process map depicts the treatment course of a patient with a large vessel occlusion (LVO) stroke. From entry at the spoke non-thrombectomy center hospital, to the CT scanner, CT, CT angiography, and CT perfusion images (depicted in the upper right corner for a patient with a right middle cerebral artery syndrome) are uploaded to the Viz.ai cloud, and then an automated mobile device notification is made to the stroke specialist if a LVO is detected by the software analysis. The stroke specialists can view and manipulate the images in 3D. By assessment of the images and the clinical history, they decide within the mobile application to activate consultations and/or transfer to a stroke thrombectomy center. Image courtesy of Viz.ai.

Viz.ai uses a CNN algorithm to automatically detect stroke patterns of LVOs. If a LVO is detected, the output of the CNN automatically alerts the stroke treatment team, without any action required by the provider who ordered the radiologic scan. The alert is sent via a mobile phone application compliant with the Health Insurance Portability and Accountability Act to the emergency room provider, neurologist, and the neurointerventional surgeon. This immediate communication demonstrated faster notifications, saving on average 52 min before LVO treatment.30 Within the app, the provider can activate emergency transport systems for ET (figure 4).

Discussion and conclusions

AI in acute stroke diagnostic imaging uses different ML algorithms for a variety of tasks, including identification of hypodensities on non-contrast CT scans and LVOs on CTA scans. The ML systems encompass mostly unmodified input abstraction input into a RFL or CNN function approximation. Reported diverse ML performance and accuracy measurements, however, limit comparison between algorithms. Many systems are evaluated against different standards on different quality input data, leading to inconsistent and potentially biased differences in accuracy metrics.

Of the available studies, comparisons were made using the reported data. ASPECTS’ comparative accuracy with humans when the gold standard is different, either non-contrast CT or MRI with DWI, may be similar or better, depending on the AI algorithm. Specifically, RFL algorithms for ASPECTS were used in over 10 studies for validation, with nearly 70% sensitivity and over 80% specificity as compared with humans (table 2). Analysis of CNN algorithms for LVO detection were limited as data were extracted primarily from study abstracts. Systems using CNNs for LVO detection report performance metrics on average 8–10% greater than those of ML employing RFL, up to 85% mean sensitivity for automatic LVO detection. Notably, core and perfusion studies from RAPID CT and MR have the highest metrics for AI accuracy, >80%, with some datasets showing 100% sensitivity to predict favorable perfusion mismatch.18 39

However, diagnosis of acute stroke by AI has not yet been perfected and errors still exist (table 2). AI algorithms in the peer-reviewed literature report a broad sensitivity metrics, with a mean of 68%, suggesting that some AI algorithms may miss up to one in three findings on imaging output (table 2). The reported reasons for failure of the algorithms are often similar. These include radiologic scan abnormalities from pre-existing central nervous system injury, inadequate contrast boluses, inability to correct for patient motion, and tortuous vessels that preclude evaluation of the contrast column.12 14 24 Despite this, some versions of AI have good predictive power to increase standardized accurate stroke diagnosis and LVO triage. In such cases, AI used to assist in interpretation of traditional stroke imaging outputs may reduce false-negative human errors in image interpretation, increase efficiency of stroke triage, and therefore minimize neuronal death and long-term morbidity and mortality.

Comprehensive platforms by iSchemaView, Viz.ai, and Brainomix use different AI methods yet can automatically detect the presence of LVOs and, with the Viz.ai software, also instantly activate emergency LVO stroke treatment teams. The benefit of these AI algorithms is seen by improved sensitive stroke detection, as an aid to physician decision-making, and by acting as a catalyst for timely LVO detection. Yet lower specificities reported by some algorithms, in the 50% range, indicate that critical review of the radiological images by a physician remains important. As the current LVO detection methods aim for higher positive detection rates, the burden on the physician to navigate an increased number of scans containing false positives will also be noticed.

Additional validation studies for LVO diagnostics are rapidly developing as ML algorithms improve. It is important to recognize that FDA and CE mark approval of these algorithms allows them to be used by trained professionals, physicians, or medical technicians, and not as a substitute for human decision-making. Therefore, in clinical practice we predict that use of the AI software may assist clinicians in their detection accuracy of ischemic stroke.

Finally, among the AI software platforms commonly used in healthcare institutions, iSchemaView RAPID, Viz LVO and Viz CTP, General Electric, and Brainomix, each provide similar features (figure 5, online supplementary table II). Among these, AI for LVO detection is in different stages of validation. Some features differ significantly based on the software platform. iSchemaView and Brainomix AI (indirect form) do not directly detect LVOs, but infer LVO presence based on asymmetry in collateral blood vessel density. Brainomix (direct form) and Viz.ai offer direct LVO detection, though only Viz.ai has reported validation metrics. AI for both direct LVO detection and automatic emergency LVO treatment system activation is available only through the Viz.ai platform.

• Download figure
• Open in new tab
• Download powerpoint

Figure 5

Software comparison acute stroke diagnostic and triage-capable software that incorporate artificial intelligence (AI) for acute stroke imaging and emergency treatment system activation. ASPECTS, Alberte Stroke Program Early CT Score; CTA, CT angiography; CTP, CT perfusion; HIPAA, Health Insurance Portability and Accountability Act; LVO, large vessel occlusion; PACS, picture archiving and communication system.

This study and systematic review have some limitations. In contrast to AI in ASPECTS and perfusion imaging, the emerging AI technologies in the past 1–2 years applied to direct and indirect LVO detection have only been described in abstracts and posters. No rigorous comparison or systematic studies have been published. Reporting of AI in LVO detectionthus required evaluation of non-peer reviewed abstracts, posters, FDA, CE material, which permits exposure and reporting bias. The selected statistical comparisons and metrics reported in each study reviewed also vary. ML systems are evaluated against different standards of different qualities on different data, and thus metrics of sensitivity and specificity are not generalizable between ML systems. There is a critical need for a clear definition of ‘ground truth’ against which algorithms are evaluated consistently. Having a consistent set of metrics is essential to improve ML in acute stroke care.

Overall, AI applied to stroke diagnosis and LVO detection has the potential to standardize and improve care. There remains a paucity of randomized controlled trials comparing AI software. Systematic and standardized methods for validation and comparison of this class of tools are needed.