Article Text
PDF
Abstract
Background The relationship between the interventionist’s experience and outcomes of endovascular thrombectomy (EVT) for acute ischemic stroke of the anterior circulation, is unclear.
Objective To assess the effect of the interventionist’s level of experience on clinical, imaging, and workflow outcomes. Secondly, to determine which of the three experience definitions is most strongly associated with these outcome measures.
Methods We analysed data from 2700 patients, included in the MR CLEAN Registry. We defined interventionist’s experience as the number of procedures performed in the year preceding the intervention (EXPfreq), total number of procedures performed (EXPno), and years of experience (EXPyears). Our outcomes were the baseline-adjusted National Institutes of Health Stroke Scale (NIHSS) score at 24–48 hours post-EVT, recanalization (extended Thrombolysis in Cerebral Infarction (eTICI) score ≥2B), and procedural duration. We used multilevel regression models with interventionists as random intercept. For EXPfreq and EXPno results were expressed per 10 procedures.
Results Increased EXPfreq was associated with lower 24–48 hour NIHSS scores (adjusted (a)β:−0.46, 95% CI −0.70 to −0.21). EXPno and EXPyears were not associated with short-term neurological outcomes. Increased EXPfreq and EXPno were both associated with recanalization (aOR=1.20, 95% CI 1.11 to 1.31 and aOR=1.08, 95% CI 1.04 to 1.12, respectively), and increased EXPfreq, EXPno, and EXPyears were all associated with shorter procedure times (aβ:−3.08, 95% CI−4.32 to −1.84; aβ:−1.34, 95% CI−1.84 to −0.85; and aβ:−0.79, 95% CI−1.45 to −0.13, respectively).
Conclusions Higher levels of interventionist’s experience are associated with better outcomes after EVT, in particular when experience is defined as the number of patients treated in the preceding year. Every 20 procedures more per year is associated with approximately one NIHSS score point decrease, an increased probability for recanalization (aOR=1.44), and a 6-minute shorter procedure time.
- stroke
- thrombectomy
- intervention
Data availability statement
No data are available. In compliance with the General Data Protection Regulation, source data underlying this article cannot be shared publicly. Information about analytic methods, R scripts/output files will be shared on reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Introduction
Increasing volumes of patients with acute ischemic stroke (AIS) treated with endovascular thrombectomy (EVT) by an intervention center are associated with better outcomes and faster procedures.1 This effect on outcomes is attributed to experience and probably mediated by faster procedures, although some researchers speculate that this effect could also be due to a selective referral pattern. However, it is unclear whether this volume–outcome relation also applies when experience is measured on the level of the individual interventionist.
Previous studies on interventionist’s experience in EVT for AIS showed shorter procedure times,2–4 higher first pass reperfusion rates,5 and a higher probability for recanalization in patients treated by more experienced interventionists.6 However, none or only small increases in the probability of good outcomes were found for patients treated by more experienced interventionists.3 6 7 This might, in part, be due to the use of long-term outcome variables, which are influenced by many other factors than just the quality of the procedure.
The relatively small effect sizes found in previous studies could also be partly explained by the chosen definition of experience. All but one of the above-mentioned studies defined experience as the total number of previously performed procedures. However, there are other ways in which to define an interventionist’s experience—for example by the frequency of procedures performed in the preceding year, or by years of experience. Knowledge about the association of these different definitions and patient outcomes is limited. Compared with the literature on EVT for AIS, the frequency of procedures performed has more frequently been used to define experience in percutaneous coronary intervention (PCI) studies.8–10 As volume (thresholds) are used as quality indicators, it is relevant to know which definition of experience best reflects an interventionist’s experience.
The primary aim of this study is to assess the effect of the interventionist’s level of experience on clinical, imaging, and workflow outcomes. The secondary aim is to determine which of the three experience definitions is most strongly associated with these outcome measures.
Methods
Study design and participants
We used data from 2700 patients included in the MR CLEAN Registry: a national, multicenter, prospective, observational monitoring study. Between March 2014 and November 2017, all patients in the Netherlands with an AIS who received an arterial puncture to undergo EVT were enrolled in the study. Detailed study design and methods have been described previously.11
For the current study, we included patients aged ≥18 years; treated in a MR CLEAN trial center; with a symptom onset to arterial puncture time ≤6.5 hours; with an AIS due to a proximal intracranial vessel occlusion as confirmed by CT angiography, magnetic resonance angiography, or digital subtraction angiography (DSA) of the anterior circulation (ie, internal carotid artery (ICA), internal carotid artery terminus (ICA-T), middle (M1/M2) cerebral artery, or anterior (A1/A2) cerebral artery). We excluded patients who underwent groin puncture but did not receive EVT. All patients were treated according to prevailing Dutch national guidelines.
All imaging was assessed by an independent imaging core laboratory, which was blinded to clinical findings, except the occlusion side.
Outcomes
The primary outcome was baseline-adjusted short-term neurological outcome scored by the treating physician using the National Institutes of Health Stroke Scale (NIHSS) score 24–48 hours after EVT. We used this outcome because it has a more direct relation with the intervention and is probably a more sensitive indicator for treatment effect than the modified Rankin Scale (mRS) score after 90 days, which is influenced by many other variables. In addition, the NIHSS score at 24–48 hours is strongly associated with the mRS score at 90 days.12 For comparison, we also calculated the effect estimate for the association between center volume and our primary outcome.
Secondary outcomes were mRS score and mortality at 90 days post-EVT13; degree of reperfusion post-EVT, as scored on the extended thrombolysis in cerebral infarction (eTICI) score14; successful recanalization (eTICI ≥2B); number of attempts; duration of procedure (groin puncture until vessel closure, as reported by the interventionist); procedural complications scored on DSA (ie, evidence of intracranial hemorrhage (ICH), dissection, perforation, vasospasm, a distal thrombus, and an embolus in new vascular territory); any serious adverse effects (SAEs); progression of ischemic stroke (defined as a decrease of at least four NIHSS points); new ischemic stroke; and symptomatic ICH. A symptomatic ICH was defined as an ICH on follow-up imaging (classified according to the Heidelberg criteria) related to neurological decline of at least four NIHSS points or death.15
Definitions of experience
We defined the level of experience of individual interventionists who performed the EVT in three different ways: the total number of previously performed EVTs (EXPno), the number of EVT procedures performed in the preceding year (EXPfreq), and years of experience since the first EVT (EXPyears). All these experience variables were calculated for each case at the moment of the current EVT. Center volume, defined as the number of patients treated with EVT in a center in the preceding year, was also calculated for each procedure.
To calculate the experience parameters for each interventionist we used all EVTs they performed (as first, second, or third interventionist on a procedure) in the MR CLEAN Registry (2014–2017; including posterior strokes), MR CLEAN pretrial (2005–2010), and the MR CLEAN trial (2010–2014).16 17 Experience outside these studies (unregistered EVTs), as reported by interventionists in response to a survey, was rare, but also taken into account to assess experience more accurately. The date of an interventionist’s first intervention could in most cases be derived from the data. When this was not reliable due to unregistered EVTs, interventionists were asked to report this date.
When more than one interventionist was involved in the procedure, data from the highest-ranking interventionist (based on EXPno) were used to estimate the association between experience and outcomes. This was done because we assumed that the most experienced interventionist had the greatest influence on the procedure.
Missing data
Missing NIHSS scores were scored based on the reported neurological examination in medical records. When scores were missing because a patient died before assessment (24–48 hours post-EVT) we assigned the maximum NIHSS score of 42 points. Ultimately, for 189 patients (7.0%), no follow-up NIHSS score was available. In 100 of 2700 (3.7%) cases, the interventionist could not be identified, and thus the experience parameters were missing for these procedures. An EXPno of ‘1’ for an interventionist performing an EVT without supervision was regarded as highly unlikely, and was probably due to failure to register the supervisor in these 11 cases. We, therefore, considered these values missing.
All missing values were imputed using multiple imputation chained equations, based on relevant covariates and outcomes.
Statistical analysis
We used crude data to report baseline and procedural characteristics. For descriptive purposes, we categorized the level of experience based on the median EXPno (which was 45 procedures) as lower (<45 procedures) and higher (≥45 procedures) experience levels. To estimate the effect of the experience parameters on the predetermined outcome variables, we used multilevel (ie, generalized linear mixed effects) models with interventionists as a random intercept to adjust for data clustering. Because nearly all interventionists operated in just one center, additional adjustments for center were regarded redundant. This assumption was supported by our sensitivity analysis. For the model of the effect of center volume on our primary outcome we used center as a random intercept to correctly adjust for clustering in the data.
We used the NIHSS score at 24–48 hours as a continuous variable. For all continuous outcome variables, we used linear models and presented beta coefficients (β) with 95% confidence intervals (95% CI). For binary and ordinal outcomes, we used respectively binary and ordinal logistic models. We presented odds ratios (ORs) with a 95% CI for the binary outcomes, and the common OR (cOR) with a 95% CI for the ordinal outcomes.
We adjusted all outcomes for age, sex, collaterals, NIHSS score at baseline, Alberta Stroke Program Early CT score at baseline, pre-stroke mRS score, pre-stroke eTICI score, occlusion segment, onset-to-groin time, intravenous thrombolysis administration, transfer from a referral hospital, diastolic and systolic blood pressure. Additional adjustments depended on the outcome variable, and are (if applicable) specified per outcome variable in the tables. Confounders were chosen based on expected associations with the outcome variables, and we also included variables that showed a significant difference at baseline between experience groups (p value ≤0.05; table 1).
Baseline characteristics
Effect estimates were expressed per 10 procedures for EXPno, EXPfreq, and center volume. For the variable EXPyears, the unit of measurement was 1 year. The mRS scores were reversed to get cORs above 1.0 for better outcomes in concordance with ORs for binary outcomes. All analyses were performed using R statistical software, version 4.0.2.
Results
Participants
Between March 2014 and November 2017, a total of 3637 patients were enrolled in the MR CLEAN Registry. For our analysis, we used the data from 2700 patients that met our criteria; the flow chart is presented in figure 1.
Flow chart of included patients.
Baseline characteristics are shown in table 1. Patients treated by interventionists with higher experience levels were referred from a primary stroke center more often (61% vs 51%), were treated with intravenous thrombolysis less often (73% vs 77%), had an M1 occlusion less often (56% vs 62%), and had an M2 occlusion more often (15% vs 11%) than patients treated by interventionists with lower experience levels. Online supplemental table I displays the procedural characteristics. Interventionists with higher experience levels used a balloon guide catheter more often (69% vs 60%), an aspiration device less often during their first attempt (25% vs 30%), and placed an ICA stent less often (5.3% vs 8.2%) than less experienced interventionists.
Supplemental material
Interventionists
Experience of 110 interventionists was calculated. Procedures were performed by a single interventionist in 1602 (62%) of 2600 cases. In 922 (36%) cases two interventionists participated in the procedure, and in 76 (3%) cases three interventionists were involved. In 85% of cases, the interventionist registered as first interventionist was also the interventionist with the highest EXPno. The median EXPno was 45 (range 2–179). The median EXPyears was 4.75 (range 0–23). The median EXPfreq was 19 (range 1–70).
Effect of experience on clinical outcomes
We observed a significant association between EXPno and NIHSS scores in univariable analysis (β −0.14, 95% CI −0.25 to −0.03). This association was no longer significant after adjustments (aβ −0.09, 95% CI −0.20 to 0.01)(table 2). A higher EXPfreq was significantly associated with lower NIHSS scores 24–48 hour post-EVT in both univariable and multivariable analyses (adjusted β (aβ) −0.46, 95% CI −0.70 to −0.21) (table 3). In other words, with every 20 procedures more per year, NIHSS scores decreased by approximately one point.
Effect estimates for the association between interventionist’s experience number (EXPno) per 10 procedures and clinical, radiological, and workflow outcomes
Effect estimates for the association between the interventionist’s experience in the preceding year (EXPfreq), per 10 procedures, and clinical, radiological, and workflow outcomes
A higher center volume was also significantly associated with lower NIHSS scores 24–28 hour post-EVT (aβ −0.18, 95% CI −0.28 to −0.08).
Higher EXPyears were associated with higher NIHSS scores in univariable analysis (β 0.15, 95% CI 0.03 to 0.26), but this relationship was no longer significant after adjustments (aβ 0.10, 95% CI −0.01 to 0.22; online supplemental table II).
After adjustments, none of the experience parameters were associated with mRS scores or mortality at 90 days (tables 2 and 3, and online supplemental table II).
Effect of experience on clinical safety outcomes
Higher EXPfreq was associated with decreased probability for the occurrence of any SAEs (adjusted OR (aOR) 0.88, 95% CI 0.82 to 0.95; table 3). Higher EXPfreq was also associated with decreased probability for stroke progression (aOR=0.88, 95% CI 0.80 to 0.98; table 3).
Effect of experience on imaging outcomes
We found an association between higher EXPno and EXPfreq and recanalization rate (aOR=1.08, 95% CI 1.04 to 1.12, and aOR=1.20, 95% CI 1.11 to 1.31, respectively; tables 2 and 3). This means that an increase of 20 procedures per year increased the probability for recanalization with an OR of 1.44. There was no association between EXPyears and recanalization (online supplemental table II).
Both higher EXPno and EXPfreq were associated with better postinterventional eTICI scores (acOR=1.05, 95% CI 1.02 to 1.08 and acOR=1.15, 95% CI 1.07 to 1.23, respectively; tables 2 and 3). There was no association between EXPyears and eTICI scores (online supplemental table II).
Effect of experience on safety outcomes scored on angiography
Higher EXPyears were associated with a lower probability of vasospasms (aOR=0.94, 95% CI 0.88 to 0.99; online supplemental table II). When adjusted for covariates a higher EXPfreq was associated with the occurrence of a distal thrombus (aOR=1.12, 95% CI 1.02 to 1.22; table 3). None of the experience parameters was associated with any of the other complications scored on DSA images (tables 2 and 3, and online supplemental table II).
Effect of experience on workflow outcomes
EXPfreq, EXPno, and EXPyears were all associated with shorter procedures (aβ −3.08, CI −4.32 to −1.84, aβ −1.34, 95% CI −1.84 to −0.85, and aβ −0.79, 95% CI −1.45 to −0.13, respectively; tables 2 and 3, and online supplemental file 1II). This means that an increase of 20 procedures per year led to an approximately 6-minute decrease in procedure time. However, no significant associations were found between any of the experience parameters and the number of attempts by the interventionists during the procedures (tables 2 and 3 and online supplemental table II).
Sensitivity analysis
We repeated analyses for the effect estimates of EXPno, EXPyears, and EXPfreq on NIHSS scores using three-level models with both the interventionist and the center as random intercepts. This did not change any of the conclusions, and effect sizes were similar.
When we used the experience parameters of the first interventionist instead of the most experienced interventionist and repeated the analysis of the effect of EXPno, EXPfreq, and EXPyears on NIHSS scores, adjusted results were similar. Only the association between more EXPyears and worse neurological outcomes became significant, and the effect size increased from (aβ 0.10, 95% CI −0.01 to 0.22) to (aβ 0.14, 95% CI 0.03 to 0.25).
Discussion
In our study, the number of procedures performed in the preceding year (EXPfreq) was associated with better (short-term) clinical, imaging, and workflow outcomes. However, there was no significant association between the interventionist’s experience and NIHSS scores when we defined experience as the total number of procedures performed (EXPno) or years of experience (EXPyears). All experience definitions were related to shorter procedure times. EXPno was also related to a higher recanalization rate and to better eTICI scores after EVT, but to a lesser extent than EXPfreq. Center volume was also associated with lower NIHSS scores 24–48 hour post-EVT, but this effect estimate was considerably smaller than the effect estimate of EXPfreq.
Clinical outcomes
Previous studies found no or small effects of the interventionist’s experience on clinical outcomes and/or mortality after EVT for AIS, but also after PCI, and carotid artery stenting (CAS) procedures.3 6 7 18 19 All but one of these studies defined experience as the total number of previously performed procedures, which could explain the effect sizes found. In studies where the annual experience was used to define experience, the effect sizes seem to be larger.7–10
Another reason for the relatively small effect sizes could be the use of long-term outcomes, such as mRS scores at 90 days. These long-term outcomes are subject to many influences besides the procedure itself. Therefore, short-term outcomes, such as the NIHSS scores at 24–48 hours, might be more representative of procedure quality. The MR CLEAN pretrial investigators did, however, also assess the effect of experience on the NIHSS score, but found no significant association.3 This could be partly due to the relatively small sample size and range of experience at that moment.
Safety outcomes
Previous studies found no significant associations between interventionist’s experience and the occurrence of SAEs and/or 30-day readmission in patients after EVT.3 6 7 However, for PCI the interventionist’s experience was associated with a reduced risk for (cardiac) death within 30 days,8 (in-hospital) mortality,9 10 and failure of the procedure.20 In CAS procedures, experience was also related to a risk reduction for transient ischemic attacks and 30-day occurrence of all neurological events or death.19
Imaging outcomes
The MR CLEAN pretrial investigators found no significant association between experience and recanalization post-EVT,3 yet a more recently published study did find a significant increase in the probability for better TICI scores.4 The relatively small effect size they found could again be partly due to the choice of experience definition (EXPno) because when we used EXPno, our effect size approaches theirs. Another study that used annual experience instead of EXPno found a significantly increased probability for recanalization between interventionists who performed <14 procedures/year compared with those who carried out ≥40 procedures/year. This, however, did not lead to fewer complications or better mRS scores.6
In the MR CLEAN Registry, an association was found between a higher EXPno and increased probability for first pass reperfusion. Additionally, first pass reperfusion itself was associated with an increased probability of good clinical outcome.5
Workflow outcomes
Previous studies support our results regarding the association between experience and shorter procedure times. In the MR CLEAN pretrial, the authors observed a relationship between more experience and shorter procedures in their sensitivity analysis (after including high-risk patients)3; this association was recently confirmed in a larger population.4 This is relevant since ‘time is brain’, but also because shorter procedures lead to less radiation and contrast exposure. The latter was investigated in a retrospective single-center study, which found that an interventionist’s experience in EVT for AIS was significantly associated with shorter procedures and less radiation exposure.2 This relation between increased experience levels and shorter procedures and lower contrast doses was also found in studies that investigated PCI,18–22 and CAS procedures.19
Experience definition
In concordance with our results, previous studies that defined experience as the number of procedures/year instead of the total number of previously performed procedures seemed to find larger effects sizes on clinical, safety, and imaging outcomes. However, comparison of these studies is difficult, since they investigated different procedures, used different methods, used other outcome variables, and calculated the experience per year differently. Studies we have referred to used the average annual experience or an experience number per calendar. We assessed experience more precisely by calculating the number of procedures performed in the preceding year for each procedure.
We found one other study which used both the absolute experience as well as the average annual experience to investigate the effect on postprocedural outcomes. That study on CAS procedures found that more annual experience was related to a lower frequency of stroke or death within 30 days, whereas the absolute experience number was not.23
Strengths and limitations
Because we calculated experience variables by using data from the MR CLEAN pretrial, MR CLEAN trial, and the MR CLEAN Registry, we had the opportunity to account for almost all EVT for AIS procedures performed in the Netherlands over more than a decade. Another strength of this study is the large nationwide registry used, making the results representative for clinical practice. Our observations regarding the influence of experience on clinical, imaging, and workflow outcomes, and our findings that frequency parameters are more strongly associated with these outcomes than the other experience definitions could be relevant in establishing minimum requirements for interventionists in guidelines. It could also be important in studies investigating new devices, techniques, and treatments since experience could be an important confounder.
Nevertheless, certain limitations need to be acknowledged. Because more than a single interventionist could participate in an EVT procedure, it was not feasible to reconstruct the relative contribution of each interventionist. More experienced interventionists could also supervise less experienced interventionists, and therefore the results found in our study may be an underestimation of the actual effect of experience on the analysed outcomes. The possible bias introduced due to this should be limited, though, since in the majority of cases only one interventionist performed the procedure. Furthermore, our sensitivity analysis did not show a major impact on our primary results when analyses were repeated using data from the first instead of the most experienced interventionist. Other possible sources of bias could be the experience an interventionist obtained by performing endovascular procedures elsewhere in the body, or other types of endovascular treatments (eg, endovascular coiling), for which we did not account. Bias could also be introduced by certain developments over time in, for example, device choices, patient selection, and increased numbers of referred patients. Finally, owing to the range of patient volumes investigated in this study, our results may not be entirely generalizable to situations in which interventionists treat much higher patient volumes.
Conclusions
In conclusion, higher levels of interventionist’s experience, measured by the number of procedures performed in the preceding year (EXPfreq), is associated with better neurological outcomes, higher recanalization rates, and shorter procedure times. Every 20 procedures more per year is associated with approximately one NIHSS score point decrease, an increased probability for recanalization (OR=1.44), and a 6-minute decrease of procedure times. A higher total number of previously performed procedures (EXPno) is also, but to a lesser extent, associated with an increased probability for recanalization and shorter procedure times. More years of experience (EXPyears) also showed a minimal reduction in procedure times. Since EXPfreq was the best predictor of outcomes after EVT, we recommend the use of frequency parameters instead of the more common absolute experience numbers for defining experience in future research.
Supplemental material
Supplemental material
Data availability statement
No data are available. In compliance with the General Data Protection Regulation, source data underlying this article cannot be shared publicly. Information about analytic methods, R scripts/output files will be shared on reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants. The MR CLEAN Registry was approved by the ethics committee of the Erasmus University MC, Rotterdam, The Netherlands (MEC-2014-235). The need for individual patient consent has been waived.
Acknowledgments
We thank the MR CLEAN Registry Investigators–group authors (online supplemental-List of group authors).
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors SGHO and SJdH cleaned the data and collected additional variables regarding the interventionists. SGHO analyzed the data, and wrote and revised the paper. All authors contributed to the design and interpretation of the work, and have critically revised and eventually approved the manuscript.
Funding The MR CLEAN Registry was supported by Stichting Toegepast Wetenschappelijk Instituut voor Neuromodulatie (TWIN); Erasmus MC University Medical Center; Maastricht University Medical Center+; and Amsterdam University Medical Center.
Competing interests DWJD reports funding from the Dutch Heart Foundation, Brain Foundation Netherlands, The Netherlands Organisation for Health Research and Development, Health Holland Top Sector Life Sciences & Health, and unrestricted grants from Penumbra Inc., Stryker European Operations BV, Medtronic, Thrombolytic Science, LLC, and Cerenovus for research, all paid to the institution. DWJD participates in the DSMBs of ESCAPE-NEXT (stopped July 2021) and TESLAT (without receiving payments). WHvZ received consultation fees from Stryker, Nico Lab, and Cerenovus, paid to the institution. WHvZ participates in the DSMBs of Philips’WeTrust study, Anaconda’s Solonda study, in Extremis Studies, Montpellier, all funding was paid to the institution. BJE received grants from the Dutch Research Foundation (ZonMW), Dutch Ministery of Economy (TKI-PPP), and Nico Lab, all paid to the institution. CvdL participates in the Secretary Dutch Interventional Society (unpaid). SMJvK reports participation in DSMBs of multiple studies as a statistician, none in the same clinical field. JS participates in the executive committee of the MR CLEAN MED, unpaid. MU received grants from the Dutch Heart Foundation, paid to the CONTRAST consortium. MU also received financial support for interventional workshops and travel visits paid to Microvention, Stryker, Cerenovus. PJvD received consultation fees and payments/honoraria from Stryker, paid to the institution.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.