Article Text

Download PDFPDF
Variability of results of recent acute endovascular trials: a statistical analysis
  1. Mayank Goyal,
  2. Bijoy K Menon
  1. Department of Radiology and Clinical Neurosciences, University of Calgary, Calgary, Alberta, Canada
  1. Correspondence to Dr Mayank Goyal, Seaman Family MR Research Centre, Foothills Medical Centre, 1403—29th St NW, Calgary, Alberta, Canada T2N 2T9; mgoyal{at}ucalgary.ca

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Five recent trials have shown the benefit of endovascular treatment in patients with acute ischemic stroke due to large vessel occlusion in the anterior circulation.1–5 There were a lot of commonalities between the trials. The key ones were: most patients had clinically severe ischemic stroke; most patients had small core based on imaging; all patients had neurovascular imaging to detect the presence of proximal vessel occlusion; most patients had an M1±intracranial internal carotid artery occlusion; a stent retriever was used for clot retrieval in the majority of patients.6–8 There were also commonalities in the results: all trials used the modified Rankin Scale (mRS) at 90 days and used shift analysis (EXTEND-IA was a phase IIB study with reperfusion and/or NIH Stroke Scale at 24 h as the primary outcome; however this trial also reported mRS at 90 days as their secondary analysis); all trials showed a statistically significant benefit of endovascular treatment over the control arm. The complication rates (symptomatic intracranial hemorrhage) of endovascular treatment were exceedingly low across all trials.

There were, nonetheless, some differences between these trials. The key differences were: some trials had a lot of focus on speed and workflow (ESCAPE and SWIFT PRIME), some trials used CT perfusion for patient selection (EXTEND-IA and most of the patients in SWIFT PRIME), while others used collateral imaging for patient selection (ESCAPE) or non-contrast CT and CT angiography alone (MR CLEAN).6 In addition, there were differences in geography, healthcare systems, site characteristics, rate of enrollment, age range (some trials did not allow inclusion of patients aged >80 years), use of intravenous tissue plasminogen activator (IV tPA) (in some trials all patients received IV tPA).9 There were also differences in outcome assessment, including assessment of reperfusion following endovascular treatment using the Thrombolysis in Cerebral Infarction (TICI) score versus use of the modified TICI score, and varying definitions of symptomatic intracerebral hemorrhage.

Although positive, these trials differed from each other in the absolute effect size of intervention versus control for the dichotomous 90-day outcome of mRS 0–2 vs 3–6. Effect sizes ranged from 13% to 31%.1–5 As a result, there has been speculation on whether patient selection, workflow, or healthcare setups were better in some trials than in others.

We present a simple statistical analysis of the results of the trials to show the importance of the CI around the effect size reported. We use a simple thought experiment to illustrate our point.

The primary results of the five recently published trials are shown in figure 1 as rate ratios with 95% CIs using a forest plot; pooled estimate (using a random effects model) of the effect sizes from all the trials are shown at the bottom of this figure. The figure clearly shows that the 95% CIs of the effect sizes across all trials overlap; larger trials contribute more to the robustness of the pooled estimate than smaller trials (see weightage assigned to each trial using the random effects model on the right side of the figure). Figure 2 shows the same data after exclusion of the results from the EXTEND-IA trial: note that there is no difference in the pooled estimate of effect size from figure 1. Figures 3 and 4 show the pooled results of the three trials that did not use perfusion imaging for patient selection compared with the two trials that did. These figures show that there is no significant difference in effect size favoring endovascular therapy across the five trials.

Figure 1

Forest plot showing effect sizes (rate ratios with 95% CIs) in the five recently published clinical trials showing benefit of endovascular treatment over control. Pooled estimate of effect size is generated using a random effects model.

Figure 2

Forest plot showing effect sizes (rate ratios with 95% CIs) after excluding the phase IIB EXTEND-IA trial. The pooled estimates of effect per random effects model are the same as in figure 1.

Figure 3

Forest plot showing effect sizes (rate ratios with 95% CIs) in the three trials (MR CLEAN, ESCAPE and REVASCAT) that did not primarily use CT perfusion imaging for patient selection.

Figure 4

Forest plot showing effect sizes (rate ratios with 95% CIs) in the two trials (SWIFT PRIME and EXTEND-IA) that used CT perfusion imaging for patient selection.

A hypothetical thought experiment is described below that illustrates the importance of sample sizes and CI around effect sizes when comparing trials. Imagine that, in each trial, a patient who was randomized to the endovascular arm, had successful reperfusion and early recovery was run over by a bus on the 70th day after the stroke while on his way for a physiotherapy session. This accidental event should not affect the validity of between-trial comparisons. However, we see that effect sizes start reducing dramatically in the smaller trials; with two such patients, the smallest of the trial is longer positive (shows a non-statistically significant effect size, figures 5 and 6). Nonetheless, the pooled estimate from the analysis and the effect sizes within the larger trials are consistently positive.

Figure 5

Hypothetical Forest plot showing effect sizes (rate ratios with 95% CIs) in the five recently published clinical trials assuming that one patient in the endovascular arm in each trial who does well with treatment dies due to random chance.

Figure 6

Hypothetical Forest plot showing effect sizes (rate ratios with 95% CIs) in the five recently published clinical trials assuming that two patients in the endovascular arm in each trial who do well with treatment die due to random chance.

These simple statistical analyses should lead us to conclude that there are no statistically significant differences in results between the five recent trials. Any apparent differences are likely due to randomness. We can also conclude that trials with a larger sample size have more robust results that are less likely to be affected by randomness. Moreover, the commonalities across the trials (acute ischemic stroke due to proximal vessel occlusion (diagnosed on CT angiography), exclusion of patients with a large core, and safe and high quality reperfusion using stent retrievers) are more valid than any apparent between-trial differences.

References

View Abstract

Footnotes

  • Competing interests BKM was a member of the steering and executive committee of the ESCAPE trial. MG was co-PI of the ESCAPE and SWIFT PRIME trials. MG has a licensing agreement with GE Healthcare for further development of systems of stroke diagnosis.

  • Provenance and peer review Not commissioned; internally peer reviewed.