Table 2

Ischemic stroke studies using artificial intelligence (AI) for acute diagnosis, triage, or complication prediction with performance compared with humans (2014–2019). Controls for bias and possible sources of bias for each study are shown in the online supplementary table I

StudyAimInclusion and exclusion criteriaStudy typeAI method
Accuracy metricsnConclusions
Maegerlein et al 201924 AI vs neuroradiologist ASPECTSAcute MCA with or without an LVOSCRVRFL, RAPIDTwo neuroradiologistsKappa human consensus: ~57%, AI 90%152ASPECTS by AI has better agreement to consensus score than independent neuroradiologists
Chatterjee et al 201929 CNN to detect LVOs from CTA imagesAnterior circulation stroke syndromesSCRVCNN, Viz.aiRadiology report without AISensitivity 82%, specificity 94%, PPV 77%, NPV 95%650CNNs detect LVOs with good accuracy
Öman et al 201914 CNN to detect MCA strokes from CTA source imagesStroke code patients: MCA cases and final diagnosis of non-stroke controlsSCRV; two-part AI development and validationCNN, 3D slicer, DeepMedicTwo radiologists’ manual segmentation (ASPECTS) and quantitative28 voxel scoreSensitivity 67–74%, specificity 93–96%, AUC 91–93%60CNNs can use CTA source images to detect hypodensities correlated to acute ischemic stroke
Kuang et al 201917 ASPECTS vs MRI DWIAcute ischemic stroke CT head and then MRI DWI brainSCRVRFLMRI DWI sequencesKappa 88%, sensitivity 98%, specificity 80%157RFL reliably assigns ASPECTS
Seker et al 201925 AI vs neuroradiologist ASPECTS with different CT postprocessing reconstructionAcute proximal MCA occlusionSCRVRFL, Brainomix e-ASPECTSFour radiologistse-ASPECTS performed better than residents: kappa 92% vs ~75%43e-ASPECTS performs better than radiologists with varied reconstructions of CT scans from the same patient
Olive-Gadea et al 201926 AI vs radiologist ASPECTS association with CTP infarct coreAcute proximal MCA or ICA terminus occlusions that received ETSCRVRFL, Brainomix e-ASPECTS, RAPID CTPRAPID CTP coreRadiologist vs AI ASPECTS correlation 44%, both scores correlated with CBV (rs = –0.41 vs –0.54)184Radiologist and AI ASPECTS compared with CBV value similarly predict infarct volume
Chriashkova et al 201927 AI vs radiologist ASPECTS and time to scoreAcute ischemic stroke syndromesSCRVRFL, Brainomix e-ASPECTS26 Clinicians (including 11 radiologists) with access to clinical symptoms and follow-up scansAI reduced scoring time by 34%, kappa improved from 26% to 38%64AI in e-ASPECTS improved efficiency and accuracy
Barreira et al 201813 CNN vs neurologist to detect LVOs from CTAAcute ischemic stroke with or without LVOMulticenter retrospective studyCNN, Viz.aiOne stroke neurologistSensitivity of 97% and specificity of 52%, PPV 74%, NPV 91%; overall accuracy 78%152CNN have high positive predictive value for presence of an LVO
Barreira et al 201812 CNN vs neurologists to detect LVOs from CTAAcute ischemic stroke with or without LVOThree-center retrospective reviewCNN, Viz.aiSeveral experienced stroke neurologistsSensitivity 90%, specificity 83%, accuracy 86%, PPV 82%, NPV 91%, AUC 86%875CNN AI detects more than 82% of anterior circulation LVOs
Boldsen et al 201816 AI vs radiologist for an ATLAS core strokeAcute anterior circulation stroke, MRI DWI scanMulticenter, retrospective case training, validationRFLOne imaging expertDice coefficient 0.61109AI provides reliable penumbra and core size
Kellner et al 201832 AI vs clinical scales to identify severe stroke types, including LVOAny brain neurological presentation with neurological imagingMulticenter, combined pilot and validation studyML, subtype not definedMultiple trial coordinators and cliniciansSensitivity 93%, specificity 87%, AUC 95%252Volumetric impedance phase shift spectroscopy run by ML reliably identifies severe strokes
Lucas et al 201819 CNN vs radiologist to predict CTP stroke growthPatients with a stroke, each with a full set of CTP dataSCRV AI development, validationCNNOne expert manual raterDice coefficient  0.46–0.5329CNN predicts infarct growth well
Goebel et al 201822 ASPECTS from: AI softwares vs each other vs radiologistAcute MCA ischemia, no hemorrhage or artifactsSCRV, head-to-head scoringFrontier vs Brainomix vs radiologistTwo senior radiologistsAgreement interclass correlation: human and Brainomix 0.71–0.84 vs human and frontier 0.47–0.54150High ASPECTS agreement between consensus radiologists and Brainomix
Guberina et al 201823 AI vs radiologist ASPECTS with acute and chronic strokesAcute MCA ischemia within 6 hours from symptom onset who were eligible for reperfusion therapySCRV; definite infarct core on head CT at 24 hoursRFL, Brainomix e-ASPECTSThree neuroradiologistsAI sensitivity 83% vs human 73%, specificity 57% vs 84%119RFLhas slightly higher sensitivity but worse specificity for ASPECTS
Nagel et al 201710 AI vs radiologist ASPECTSAcute anterior circulation ischemic stroke syndrome, baseline and 24–36-hour follow-up head CT scansRetrospective, multicenter comparison studyRFL, Brainomix e-ASPECTSThree radiologistsAI sensitivity ~45%, specificity ~92%, accuracy ~86%132RFL was non-inferior to neuroradiologists for ASPECTS
Lisowska et al 201728 CNN vs radiologist acute stroke hypodensity and dense vessel sign detectionAcute ischemic stroke within 6 hours from symptom onset with non-contrast CTRetrospective, pooled patients from three studiesCNN, Keras on Theano (Python)One researcher, one radiologistAI AUC 92–96%51CNN detection of asymmetrical hypodensity and dense vessel can detect ischemia well
Herweh et al 20169 AI vs neurologist ASPECTSAcute stroke, CT and MRI with DWI images <2 hours apartRetrospective single center studyRFL, Brainomix e-ASPECTSThree stroke experts and three neurology trainees, DWI as gold standardAI sensitivity 46%, specificity 95%. AI correlation coefficient 0.44 vs humans 0.19–0.3834RFL accuracy for ASPECTS was similar to neurologists
Bentley et al 201437 AI vs radiologist prediction of sICH, mRSAcute ischemic stroke treated with IV tPASCRV, moderate-supervision of AISVMRadiologist SEDAN and HAT scoringRadiologist 67% vs AI 74% AUC112AI is superior to radiologist scales for sICH
Takahashi et al 201431 AI vs neuroradiologist detection of MCA dot signAnterior ischemic stroke, first head CT within 24 hoursTwo-center, retrospective studySVMTwo neuroradiologistsAI sensitivity 98%, false positive rate 1.3% per image7AI SVM is accurate to detect the MCA dot sign
Thornhill et al 201433 AI vs neuroradiologist intraluminal thrombus from plaquePatients with acute TIA or stroke with suspected free-floating luminal thrombus on CTASCRV, training and ML validationANN, SVMTwo neuroradiologistsSVM sensitivity 88%, specificity 71%, AUC 85%.23AI is ableto distinguish shape signatures of intraluminal thrombus from plaques
  • AI, artificial intelligence; ANN, artificial neural network; ASPECTS, Alberta Stroke Programme Early CT Score; AUC, area under the curve; CBV, cerebral blood volume; CNN, convolutional neural network; CTA, CT angiography; CTP, CT perfusion; ET, endovascular thrombectomy; ICA, internal carotid artery; LINDA, lesion identification with neighborhood data analysis; LVO, large vessel occlusion; MCA, middle cerebral artery; ML, machine learning; mRS, modified Rankin Scale; NPV, negative predictive value; PPV, positive predictive value; RFL, random forest learning; SCRV, single-center, retrospective review; sICH, symptomatic intracerebral hemorrhage; SVM, support vector machine (form of artificial network); TIA, transient ischemic attack; tPA, tissue plasminogen activator.