Background Thrombus radiomics (TR) describe complex shape and textural thrombus imaging features. We aimed to study the relationship of TR extracted from non-contrast CT with procedural and functional outcome in endovascular-treated patients with acute ischemic stroke.
Methods Thrombi were segmented on thin-slice non-contrast CT (≤1 mm) from 699 patients included in the MR CLEAN Registry. In a pilot study, we selected 51 TR with consistent values across two raters’ segmentations (ICC >0.75). Random forest models using TR in addition or as a substitute to baseline clinical variables (CV) and manual thrombus measurements (MTM) were trained with 499 patients and evaluated on 200 patients for predicting successful reperfusion (extended Thrombolysis in Cerebral Ischemia (eTICI) ≥2B), first attempt reperfusion, reperfusion within three attempts, and functional independence (modified Rankin Scale (mRS) ≤2). Three texture and shape features were selected based on feature importance and related to eTICI ≥2B, number of attempts to eTICI ≥2B, and 90-day mRS with ordinal logistic regression.
Results Random forest models using TR, CV or MTM had comparable predictive performance. Thrombus texture (inverse difference moment normalized) was independently associated with reperfusion (adjusted common OR (acOR) 0.85, 95% CI 0.72 to 0.99). Thrombus volume and texture were also independently associated with the number of attempts to successful reperfusion (acOR 1.36, 95% CI 1.03 to 1.88 and acOR 1.24, 95% CI 1.04 to 1.49).
Conclusions TR describing thrombus volume and texture were associated with more attempts to successful reperfusion. Compared with models using CV and MTM, TR had no added value for predicting procedural and functional outcome.
Data availability statement
Data are available upon reasonable request. Data requests forms can be obtained at the MR CLEAN website: www.mrclean-trial.org. All code is made available on github: github.com/henkvanvoorst92/Radiomics.
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
WHAT IS ALREADY KNOWN ON THIS TOPIC
Current evidence suggests a strong relationship between thrombus radiomics and the number of endovascular retrieval attempts to successful reperfusion in patients with acute ischemic stroke.
WHAT THIS STUDY ADDS
In this multicenter cohort study including 699 patients we did not find an added value of thrombus radiomics compared to using clinical baseline variables or manual thrombus measurements for predicting successful reperfusion, number of endovascular treatment attempts to successful reperfusion, and functional outcome.
Thrombus volume and texture were independently associated with the number of retrieval attempts to successful reperfusion.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
Future thrombus radiomics research should adopt an integrated approach, considering multicenter data and a comparison with using clinical variables and manual thrombus measurements to establish clinically relevant and generalizable evidence.
Removing thrombi with endovascular treatment (EVT) is the standard of care for all eligible patients with acute ischemic stroke due to a large vessel occlusion.1 2 Thrombi vary in composition and morphology,3 resulting in a variety of thrombus shapes and textures that can be visualized on non-contrast enhanced CT (NCCT). Such thrombus characteristics might be used by interventional radiologists to guide EVT decisions, enabling further improvement of procedural and functional outcomes.
Achieving successful recanalization of the occluded artery with the lowest number of device attempts is the primary goal of EVT as the functional outcome is negatively affected by the number of attempts regardless of reperfusion status.4–7 Several thrombus imaging characteristics such as thrombus density, perviousness and length extracted from manual thrombus measurements (MTM) have been associated with reperfusion and functional outcome.8 9 Although thrombus imaging characteristics extracted from MTM are intuitive, they are inaccurate due to variable thrombus shape or inhomogeneous intensities across the thrombus.10
More complex thrombus characteristics can be quantified using radiomic features, which are based on a set of programmed rules to describe a region of interest using thrombus segmentations.11 Although potentially valuable, thrombus radiomics (TR) have scarcely been related to recanalization outcomes in patients with acute ischemic stroke.12 13 To date, only two small single-center studies in <100 patients have found an association between TR, first attempt reperfusion, and number of attempts to achieve reperfusion in patients treated with EVT.12 14 It remains unclear how well these findings generalize to other populations, whether TR have an added predictive value compared with clinical variables (CV) and MTM, and whether the effects found on procedural outcomes also affect functional outcome.
In this study we aimed to identify the added value of TR for predicting successful reperfusion, the number of EVT attempts required for successful reperfusion, and the functional outcome in a large dataset of patients treated with EVT. We compared the predictive value of TR as an additive and an alternative to the use of MTM and baseline CV. Furthermore, we assessed the effect of three shape and texture TR with the highest feature importance on reperfusion, number of attempts to successful reperfusion and 90-day modified Rankin Scale (mRS) score.
A total of 699 patients were included retrospectively from the Multicenter Randomized Controlled Trial of Endovascular Treatment for Acute Ischemic Stroke in the Netherlands (MR CLEAN) Registry, a nationwide prospective, observational, multicenter registry enrolling consecutive patients with EVT-treated large vessel occlusion acute ischemic stroke in 16 intervention hospitals in the Netherlands from March 2014 to November 2017.15 The study was exempted by an ethical committee. Individual patient data cannot be made available under Dutch law as we did not obtain patient approval for sharing individual coded patient data. All syntax files and output of statistical analyses are available on reasonable request to the corresponding author.
All eligible patients received 0.9 mg/kg intravenous alteplase (IVT) prior to EVT. The exact EVT approach and material choice were left to the discretion of the treating neurointerventionist. EVT could consist of stent retriever thrombectomy, aspiration thrombectomy, or a combined approach with or without administering additional intra-arterial thrombolytic agents. For the current study we used the following inclusion criteria: intracranial occlusion in the anterior circulation (intracranial internal carotid artery (ICA), M1, M2, M3 segments of the middle cerebral artery and A1/A2 segments of the anterior cerebral artery), age ≥18 years, onset to groin puncture time <6.5 hours, treatment in a MR CLEAN trial center, and thin-slice baseline NCCT and single-phase CT angiography (CTA) scans available acquired within 30 min on the same scanner. For patients who were transferred from a primary stroke center we used the imaging data from the primary stroke center for thrombus measurements. Furthermore, we performed a pilot study to assess consistency of TR for variations in manual thrombus segmentation based on 58 patients included in the MR CLEAN trial.16
Imaging characteristics assessed by the imaging core laboratory are detailed in online supplemental methods I.
Manual thrombus segmentations and thrombus measurements
For both thrombus segmentations and measurements, NCCT and single-phase CTA images (≤1.0 mm) were aligned with rigid co-registration using Elastix.17 We excluded patients with incorrigible co-registration misalignment between NCCT and CTA, severe noise, beam hardening artefacts, or other image quality issues preventing accurate thrombus segmentation.
Thrombus segmentations and measurements were performed in ITK-SNAP by a team of 14 trained raters: HV, AAEB, WY, JA, EW, BGD, PK, NAT, JWH, MT, MK, JB, NB, KRK (online supplemental figure S1). Thrombus segmentations in the context of our pilot study were performed by AAEB and HV. Raters were blinded to all clinical information except for symptom side. All raters received a 2-day training session and regular consensus readings were held under supervision of two neuroradiologists with >15 years of experience (CBLMM and BJE). For thrombus segmentations, the extension of the thrombus was assessed on single-phase CTA. If thrombus borders could not be clearly identified on CTA, the extension of the thrombus was based on the hyperdensity on the co-registered NCCT.8
Thrombus density and perviousness measurements have been described before.8 18 Thrombus length was measured by placing five regions of interest (ROIs) with a 1 mm radius: one at the proximal thrombus border, one at the distal thrombus border and three ROIs in the proximal, middle and distal part of the thrombus. Thrombus length in mm was computed as the path length between proximal and distal thrombus border and included the three in-thrombus ROIs. Similar to thrombus segmentations, thrombus borders were primarily assessed on single-phase CTA.
Four types of variables were defined: baseline CV, thrombus imaging characteristics derived from MTM, TR features measured in NCCT derived from manual segmentations, and outcome measures. Online supplemental table S1 shows an overview of all variables and their definitions. The CV, MTM, and TR variables are used as explanatory input variables for a random forest (RF) model predicting the outcome measures. RF is a popular machine learning algorithm due to its ability to handle different types of data (continuous and categorical), correlated variables, and variables with interaction effects.19 Furthermore, RF is a computational efficient algorithm for which the relative contribution of each variable to the models’ predictions can be computed by means of the feature importance (FI).19 Initially we explored the use of XGBoost but we encountered issues with severe overfitting. For each target variable, six experiments were conducted with deviating input variables used to optimize an RF: only CV, only MTM, only TR, CV with TR, CV with MTM, CV with MTM and TR. Before the final RF prediction model development, the most valuable input variables were selected. Variable selection as well as RF development were performed on a randomly selected training set of 499 patients, while evaluation metrics were reported based on a separate test set of 200 patients. FI of all variables was computed based on the RF model trained with mRS ≤2 as outcome using CV, MTM, and TR as input variables. Three shape and three texture TR with the highest FI were related to eTICI, number of attempts to successful reperfusion, and 90-day mRS using ordinal logistic regression.
Statistical analysis and evaluation metrics
RF classifiers were optimized to predict binary outcome measures: successful reperfusion (eTICI ≥2B), first attempt eTICI ≥2B, eTICI ≥2B within three attempts, and functional independence defined as mRS score ≤2 at 90 days follow-up. Successful reperfusion within three attempts was added as a target variable since the benefit of more than three attempts is uncertain.20 RF models were primarily evaluated with the area under the receiver operating characteristic curve (AUC). Relationships of TR were described with univariable (OR) and (adjusted) common odds ratios ((a)cOR) with 95% CI. Statistical adjustment was performed for age, sex, time from onset of neurological deficit to groin puncture, history of atrial fibrillation, occlusion location, and baseline National Institutes of Health Stroke Scale score. The number of attempts to successful reperfusion was set to a range of 1–5, >5 attempts with successful reperfusion was set to 5, and poor reperfusion (eTICI <2B) was set to 6. We compared CV and MTM variables of our train and test set combined with the not-included MR CLEAN Registry population using the χ2 test, ANOVA, and the Kruskall–Wallis test for categorical, normally distributed continuous variables, and non-normally distributed continuous variables, respectively. Missing values of CV and MTM were imputed with single imputation based on multivariable imputation with chained equations using R (version 3.6.3).
Input variable selection
We conducted a pilot study using PyRadiomics version 3.0.1 to extract 107 TR from previously collected and segmented thrombi.21 The two-way random effects intraclass correlation coefficient (ICC) was computed to assess consistency of TR. TR with poor or moderate consistency (ICC <0.75) were excluded from further analyses.22 23
All continuous variables in the training and testing set, including TR, were normalized by subtracting the training set median and dividing by the training set IQR. This normalization counteracts the intrinsic preference of RF to variables with high variance.24 The following steps were repeated for each target variable. First, a RF was trained on all variables. Tuning of RF settings (hyperparameters) was performed with 10-fold cross-validation and based on mean AUC over the validation splits. Online supplemental table S2 gives an overview of the hyperparameters considered. Variables with negligible added value (FI <0.001) were then excluded. Subsequently, a Spearman correlation matrix was constructed considering all input variables. If two variables had a correlation >0.6, the variable with a lower FI was excluded. A final RF was trained on the selected variables and its three shape and texture TR with the highest FI were selected for ordinal regression analyses on the test set. Figure 1 provides an overview of the modelling and variable selection procedures. Python code used for analytical methods is made available from: github.com/henkvanvoorst92/ThrombusRadiomics.
We included 699 patients from the MR CLEAN Registry (see online supplemental figure S2). Baseline and treatment characteristics of these patients are summarized and compared with the remainder of the MR CLEAN Registry population in online supplemental table S3. In the included testing and training set, fewer patients received IVT, patients had shorter onset to groin puncture times, fewer retrieval attempts, and a higher eTICI score compared with the remainder of the MR CLEAN Registry population.
From the 58 segmentations performed in the context of our pilot study, 107 TR were extracted from NCCT. Of these TR, 51 had an ICC ≥0.75 while 56 had an ICC <0.75. Online supplemental table S4 shows the ICC of all TR.
Table 1 shows all the test set AUC results per experiment performed. All RF models showed poor to moderate predictive performance for successful reperfusion, first-pass eTICI ≥2B, and eTICI ≥2B within three attempts, with AUC not exceeding 0.61 regardless of the variable set used. Predictive performance for functional independence was 0.81 for the model with CV as input variables. Online supplemental tables S5 and S6 show the AUC of the training set and a more extensive set of evaluation metrics for each RF model. A fall in AUC between training and test set was observed especially for the first-attempt eTICI ≥2B model but also to a lesser extent for the other models when more variables were added. This indicates a degree of overfitting.
Association between TR and outcome measures
Table 2 shows ORs based on the test set (n=200) for the six TR with the highest FI. Voxel volume was associated with more retrieval attempts to eTICI ≥2B (acOR 1.36, 95% CI 1.03 to 1.88) and worse functional outcome in the unadjusted analysis (cOR 0.75, 95% CI 0.58 to 0.94). Furthermore, the most important textural feature (inverse difference moment normalized) was significantly associated with more retrieval attempts to eTICI ≥2B (acOR 1.24, 95% CI 1.04 to 1.49) and after adjustment for confounders a relationship with eTICI was also detected (acOR 0.85, 95% CI 0.72 to 0.99). Other textural features (dependence variance and large dependence emphasis) were also associated with more retrieval attempts to eTICI ≥2B, but these relationships did not hold after adjustment for confounders. Figure 2 shows an example of two thrombi with low and high textural homogeneity scores.
In this multicenter retrospective cohort study we identified two TR that were related to our outcome measures in patients with anterior circulation stroke: thrombus volume and texture were independently associated with more attempts to eTICI ≥2B. Specifically, the identified TR texture feature (inverse difference moment normalized) describes local homogeneity of the thrombus, with a higher value implying more regions with similar Houndsfield unit values and a lower value being indicative of a more granularly textured thrombus. We did not find added value of TR over CV or MTM for predicting the number of attempts required to achieve successful reperfusion, successful reperfusion rate, and functional outcome at 90 days.
Previous studies have found that TR could be used to predict reperfusion with IVT,13 first attempt reperfusion,14 and the number of EVT attempts required for successful reperfusion, respectively.12 In contrast to our work, two small single-center studies found that thrombi with a more heterogeneous texture and higher density values were associated with more attempts required for successful reperfusion and a lower chance of first attempt reperfusion.12 14 The higher IVT percentages in our study might have contributed to this difference because our results were in accordance with a previous radiomics study performed in IVT-treated patients.13 Hofmeister et al achieved a much higher AUC (0.88) for first-pass reperfusion classification than we did (AUC=0.61) and they also found an effect related to thrombus density measures. This study used data from a single scanner in a single center, deriving many more TR (n=1485) than patients. These two methodological approaches are known to be prone to model overfitting, resulting in potential overestimation of the predictive performance and false positive findings with respect to the value of specific TR.25 26 We have chosen a stricter analytical regime and used multicenter data to reduce the probability of false positive findings. By reducing the number of TR analyzed through our selection methods, we prevented overfitting of the RF models. Due to the large number of thrombi in our cohort, we were able to split our data into a separate training set for TR selection and optimization of the RF model and a strictly separate testing set for evaluation. We believe that future radiomics studies can benefit from our pilot study by reusing our reported selection of consistent TR values.
The findings from this study only consider predictive values and associations of TR with EVT procedural and functional outcomes, while TR extracted from thrombi of patients with acute ischemic stroke could also be used to predict the most beneficial treatment approach for each patient. For example, the choice to administer IVT prior to EVT or the first-line device choice (aspiration or stent retriever) could potentially be informed by TR. In addition, thrombus textural information captured by TR may reflect histopathological characteristics. Currently, fibrin-rich thrombi (non-hyperdense appearance on NCCT) have been found to be more difficult to retrieve with EVT in some studies.27 28 It is possible that certain TR can also also be related to certain histopathological characteristics and resistance to retrieval with EVT. More insight into this relation could be useful for device selection in EVT and for the development of new treatment strategies. As such, the potential of TR for individualized treatment decisions and possible associations between TR and histopathological characteristics needs to be further investigated.
Our study is first limited by addressing the consistency of TR over multiple expert-based segmentations in which we did not consider consistency across different CT scanners, vendors, and post-processing methods. Poor reproducibility of radiomic feature values across different scanners is a known problem of radiomics, making it difficult to extrapolate findings.23 Second, full thrombus segmentations are time consuming, preventing clinical implementation of TR. However, recently, an automated thrombus segmentation has been proposed.29 The segmentations obtained by this method had high spatial overlap with expert based ground truth annotations. If this spatial overlap also results in consistent TR values, this method could be used in future studies and could be a first step to clinical implementation of full thrombus segmentations. Third, the extent of the thrombus is probably overestimated in our study because we used single-phase CTA for thrombus measurements and segmentations. The use of 4D-CTA (reconstructed from a whole brain CT perfusion scan) would have allowed for more accurate measurements,30 but CT perfusion was not included in the standard imaging protocol during enrolment in the MR CLEAN Registry. Fourth, we had to exclude a large number of patients because of unavailability of thin-slice imaging. In clinical practice, scans are acquired in thin slices but are reconstructed into thicker slices to suppress noise and reduce the amount of data and processing times. As data storage facilities improved over the years, in the later years of the MR CLEAN Registry storage of thin-slice imaging became more common. Furthermore, thin-slice imaging is more often stored in high-volume intervention centers compared with smaller comprehensive stroke centers. This may have caused an inclusion bias and might reduce the generalizability of our results. Fifth, FI scores of correlated variables are lower due to the RF optimization process; since we used FI to exclude one of two correlated variables, this might have resulted in omitting variables with a significant relationship to the presented outcome measures.
In this multicenter retrospective cohort study, TR describing thrombus volume and texture were associated with more attempts to successful reperfusion in patients with anterior circulation stroke. These TR, however, were not related to functional outcome. Compared with models using clinical variables and manual thrombus measurements, TR had no added value for predicting successful reperfusion, first attempt reperfusion, reperfusion within three attempts, and functional outcome.
Data availability statement
Data are available upon reasonable request. Data requests forms can be obtained at the MR CLEAN website: www.mrclean-trial.org. All code is made available on github: github.com/henkvanvoorst92/Radiomics.
Patient consent for publication
This study involves human participants but IRB waiver was given by the Erasmus MC Medical Ethical Committee (Institutional Review Board). This study was conducted as a retrospective analysis of a quality registry and was therefore exempted from ethical committee approval. IRB waiver was given since patients were included as part of a prospective quality registry.
HvV and AAEB contributed equally.
Collaborators on behalf of the MR CLEAN Registry collaborators. All group authors and affiliations are shown in online supplemental file 2.
Contributors HvV, AAEB: Planning, conceptual thinking, data collection and excecution. WY, JA, EW, BGD, PRK, NAT, JWH, MLT, MK, JB, NB, KRvK, MSK. Data collection, annotation, and proof reading. HMH, MK, WHvZ, HMMvB, AvdL, BJE, HAM, YBWEMR, MWAC, CBLMM: Reading and revision. BJE, HAM, YBWEMR, MWAC, CBLMM: Supervision. HvV is the guarantor.
Funding The MR CLEAN Registry was partly funded by the TWIN Foundation, Erasmus MC University Medical Center, Maastricht University Medical Center, and Amsterdam UMC.
Competing interests Amsterdam UMC received funds from Stryker for consultations by CBLMM and YBWEMR. Unrelated to this study, Amsterdam UMC received grants from the Netherlands Organization for Health Research and Development, Health Holland Top Sector LSH and Nicolab. Dr Dippel and AvdL report unrestricted grants from Stryker, Penumbra, Medtronic, Cerenovus, Thrombolytic Science, LLC, Dutch Heart Foundation, Brain Foundation Netherlands, The Netherlands Organization for Health Research and Development, and Health Holland Top Sector Life Sciences & Health for research, paid to institution. Maastricht University Medical Center received funds from Stryker, Cerenovus, Nicolab and Philips for consultation by WHvZ. NAT received funding from the AMC Medical Research. PRK is funded by INSIST: a European Union’s Horizon 2020 research and innovation program. YBWEMR is minor shareholder of Nicolab. HAM is co-founder and shareholder of Nicolab. CBLMM reports a grant from the TWIN Foundation. Unrelated to this study, CBLMM is a minor shareholder of Nicolab and reports grants from CVON/Dutch Heart Foundation, European Commission, Health Evaluation Netherlands, and Stryker, all paid to institution.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.