Background and purpose To investigate the relationship between physician and site experience and the risk of 30 day hemorrhagic and ischemic strokes in the stenting arm of the Stenting and Aggressive Medical Management for the Prevention of Recurrent Ischemic Stroke (SAMMPRIS) trial.
Methods Study records and an investigator survey were examined for physician and site related factors, including: number of Wingspan and aneurysm stents submitted for credentialing, number of study procedures performed in SAMMPRIS, years in practice after training, primary specialty, and site enrollment. Bivariate and multivariate analyses were performed to determine if these factors were associated with the 30 day rate of cerebrovascular events after angioplasty and stenting.
Results 213 patients underwent angioplasty alone (n=5) or angioplasty and stenting (n=208) with study devices by 63 interventionists at 48 sites. For credentialing, the median number of Wingspan and similar aneurysm stent cases submitted by study interventionists were 10 and 6, respectively. Interventionists with higher numbers (>10) of Wingspan cases submitted for credentialing tended to have higher rates of 30 day events (19.0% vs 9.9%) than those with <10 cases. High enrolling sites in the trial tended to have lower rates of hemorrhagic stroke (9.8% at sites enrolling <12 patients vs 2.7% at sites enrolling >12 patients).
Conclusions Interventionists credentialed with less Wingspan experience were not responsible for the high rate of periprocedural stroke in SAMMPRIS. Hemorrhagic stroke may be related to low enrollment in the trial but not previous Wingspan experience.
Statistics from Altmetric.com
Enrollment in the Stenting and Aggressive Medical Management for the Prevention of Recurrent Ischemic Stroke (SAMMPRIS) trial was stopped early, after randomization of 451 patients (planned 764), owing to the higher than expected 30 day rate of stroke or death after percutaneous transluminal angioplasty and stenting (PTAS) relative to the medical arm.1 Of the 224 patients randomized to PTAS, 33 (14.7%) suffered a stroke within 30 days of randomization compared with 13 (5.8%) in the medical arm.
Given the higher than expected rate of stroke after PTAS in SAMMPRIS, it is important to determine if technical factors, including previous experience with the Wingspan stent, training background, and the metrics chosen for the credentialing process may have contributed to the high complication rate. The purpose of the present study was to examine the relationships between factors that reflect interventionist and site experience and the risk of cerebrovascular events within 30 days (termed periprocedural events).
The SAMMPRIS trial is a randomized, prospective, multicenter, National Institutes of Health funded, blindly adjudicated trial of PTAS with aggressive medical management versus aggressive medical management alone. PTAS in the trial was performed with the Gateway PTA Balloon Catheter and Wingspan Stent System (both manufactured by Boston Scientific Corporation, now Stryker Neurovascular, Fremont, CA, USA). Details of the study design have been published previously, as well as the 30 day outcomes with follow-up out to 1 year in approximately half of the enrolled subjects.1 ,2 Medical treatment and follow-up of enrolled patients will continue until March 2013.
Interventionists interested in participating in the trial were required to submit operative reports and documentation of short term outcome (discharge summary or follow-up clinic notes) from 20 consecutive intracranial stenting or angioplasty alone cases. If the physician had not done 20 cases with the Wingspan stent, the remainder of the 20 cases could be, in order of preference, coronary stents for intracranial atherosclerotic disease, stent assisted coiling of brain aneurysms, and angioplasty alone for intracranial atherosclerotic disease. A minimum of three cases with the Wingspan stent was required for consideration.
The order of preference of non-Wingspan procedures was chosen based on the rationale that experience and documented success with placement of coronary balloon expandable stents for intracranial stenosis would predict good performance with the Wingspan stent for intracranial stenosis. Allowing self-expanding stents for stent assisted coiling of aneurysms to be included in credentialing cases was based on the fact that these stents are deployed in a similar manner to the Wingspan stent. Finally, in the uncommon scenario that an experienced interventionist did not have 20 cases of Wingspan, coronary balloon expandable stents, or self-expanding stents for the treatment of aneurysms but had experience and success with angioplasty alone for atherosclerotic intracranial stenosis, those cases were allowed for credentialing based on familiarity with the endovascular treatment of intracranial atherosclerosis and good technique with a fundamental aspect of the study procedure.
The submitted procedure notes and outcome documents for all 20 cases for every interventionist were reviewed by members of the credentialing committee (Barnwell, Derdeyn (Chair), Dion, Fiorella, Gobin, Meyers, Zaidat, Chimowitz (non-voting) and Lane (non-voting)) and abstracted for full review by the committee. Cases with complications were flagged for discussion. Committee decisions fell into three categories: approval, rejection, or deferral until more experience or better outcomes with the Wingspan stent was documented. Approval was required to be a unanimous decision. The threshold for approval, in terms of numbers of cases with the Wingspan stent, for additional physicians at the same site as an approved physician was generally lower (although never less than three), particularly if there was evidence from procedure notes of joint participation in cases.
Monitoring of interventional performance in the trial
Performance of interventionists in the trial was closely monitored to ensure patient safety and adherence to the protocol.2 Notes for all procedures performed in SAMMPRIS were reviewed by the neurointerventional principal investigators (if no procedural adverse event was reported) or the internal neurointerventionist safety monitor (if a procedural adverse event was reported). Interventionists were contacted by the co-principal interventional investigators if there were any questions raised regarding technique or protocol adherence. An interventionist was investigated if there was any suspicion by the safety monitor of poor judgment or technique, or if a safety threshold was crossed (any single occurrence of vessel rupture or more than one procedural related serious adverse event or technical problem with the study device reported in the first 10 cases).
Data used for analyses
For the purpose of the present study, we collected data from the following sources: the SAMMPRIS credentialing database, a survey sent by email to all interventionists who participated in the trial, the SAMMPRIS trial database, and the two largest published multicenter Wingspan registries in the USA.3 ,4
The following data were obtained from the credentialing database: number of cases submitted by each interventionist for credentialing with the Wingspan stent, coronary stents, aneurysm stents, and angioplasty alone for intracranial atherosclerotic stenosis. Years in practice after neurointerventional training by the start of the trial and primary specialty were collected from the interventionists’ survey.
Data derived from the SAMMPRIS trial included the number of patients enrolled in the trial at each site and the number of SAMMPRIS PTAS performed by each interventionist. Total site enrollment was used to divide the sites into high enrolling (the highest volume sites accounting for 50% of all enrolled patients) and low enrolling (the remaining sites). An analysis of individual operator experience gained over the course of the trial was not feasible, owing to the large number of operators with very few cases.
Enrollment data from the two published Wingspan registries were used to determine if a SAMMPRIS site had a principal interventionist who had been the primary operator at one of the top five enrolling sites in either of the two Wingspan registries that had mutually exclusive participating sites.3 ,4 If so, that SAMMPRIS site was considered one of the top 10 enrolling sites in the Wingspan registries for this analysis. Between the close of the registries and the beginning of SAMMPRIS, three principal interventionists had moved from another institution participating in the registries to a SAMMPRIS site that had not participated in the registries. In those situations, the site that the interventionist moved to (rather than from) was considered as a high enrolling site in the registries because it is the interventionists’ experience that we were most interested in for this analysis.
Thirty day outcomes
Of the 224 patients randomized to PTAS in SAMMPRIS, 11 patients did not undergo PTAS. Therefore, 213 patients underwent angioplasty alone (n=5) or angioplasty and stenting (n=208) with study devices. The analyses in this paper are confined to the 213 patients who underwent PTAS. Periprocedural strokes were subcategorized as hemorrhagic and ischemic, and further broken down into primarily subarachnoid (SAH) or intraparenchymal (ICH) for hemorrhage, and occlusion of local perforators for ischemic strokes. The details of these subgroup categorizations have been reported.5 Asymptomatic hemorrhagic strokes were included, as were cerebral infarctions with temporary symptoms (CITS) as mechanistically these were considered important events even though they were not primary endpoints.
For each of the factors, patients were classified into categories and the percentage of patients with an event was compared among the categories using Fisher's exact test. For continuous factors (number of credentialing cases with Wingspan stents and aneurysm stents, number of years in practice, and number of SAMMPRIS PTAS cases), two categories were formed according to the median value among the interventionists. Exact unconditional 95% CIs for the difference in percentages between groups were calculated using a score statistic.6 For the factor with three categories (specialty of the primary interventionist), a Bonferroni correction to the α level was applied. Stepwise logistic regression analysis was done to relate the occurrence of an endpoint to multiple factors. To determine if the credentialing factors were significant when accounting for the clinical factors identified in a separate analysis,5 we report the results of an analysis in which the significant credentialing factors are included in a model with the clinical factors. The p value for inclusion or removal from the model was 0.05. No adjustments were made for multiple comparisons in hypothesis tests. All analyses were done in SAS V.9.3.
Experience with device and training background
Angioplasty alone (n=5) or angioplasty and stenting (n=208) in the trial was done by 63 interventionists at 48 sites. Thirty-nine interventionists were radiologists, 18 were neurosurgeons and six were neurologists. The median numbers of procedures submitted for credentialing were 10 (range 3–20) with the Wingspan stent, 1 (range 0 to 16) with coronary balloon expandable stents, 6 (range 0–17) using aneurysm stents, and 0 (range 0 to 16) for angioplasty alone. Within the trial, the median number of PTAS procedures per interventionist was 3 (range 1–13)
Monitoring of interventionists in trial
No interventionists were suspended for safety or protocol concerns. One interventionist had two adverse events (one primary endpoint, one technical issue) in the first two PTAS cases; however the site was terminated for failing to meet enrollment goals before any additional patients were recruited.
All cerebrovascular events (n=34)
There were a total of 34 cerebrovascular events within 30 days of enrollment in patients undergoing PTAS. There were 19 ischemic strokes, two CITS, 11 symptomatic hemorrhagic strokes, and two asymptomatic hemorrhagic strokes. The 34 events occurred at 25 investigational sites. Of seven sites at which more than one of these events occurred, five were among the highest enrolling sites. Table 1 shows the 30 day rates of any cerebrovascular event according to the various interventionist and site features that were evaluated. None of the interventionist or site features was significantly (p<0.05) associated with any cerebrovascular event in bivariate analysis. The rates of any periprocedural cerebrovascular events were 9.9% for interventionists credentialed with <10 Wingspan cases versus 19.0% for interventionists credentialed with >10 Wingspan cases (p=0.11).
Hemorrhagic strokes (n=13)
There were 13 hemorrhagic strokes. Of these, seven were ICH, one of which was asymptomatic and not counted as a 30 day primary endpoint in the trial. All but one of the seven ICHs became evident or were identified within 12 h after the procedure. The mechanism of these hemorrhages was attributed to reperfusion hemorrhage as they were intraparenchymal and delayed and therefore considered unlikely to be related to wire perforation or vessel rupture. There were six SAHs. Four of the six were definite wire perforations and one was a vessel rupture. One of the six SAH patients was asymptomatic. One of the wire perforations was treated with coil occlusion of the injured branch which resulted in an ischemic stroke. This event was counted as an ischemic stroke in the primary paper but for the purpose of this analysis focusing on the initial causative mechanism, it is counted as a symptomatic SAH.
Comparisons of operator and site variables with total and subgroup hemorrhage categories are shown in table 2 The rates of any hemorrhagic stroke were 9.8% at low enrolling sites in SAMMPRIS versus 2.7% at high enrolling sites (p=0.04). The rates of any hemorrhagic stroke were 2.8% for neurosurgeons, 6.1% for radiologists, and 15.4% for neurologists (p=0.07). The rates of any hemorrhagic stroke were not related to the number of Wingspan stents credentialed with, years interventionists had been in practice, or whether a site was a high enrolling site in the Wingspan registries. In multivariate analysis the only variable that was associated with any hemorrhagic stroke was high versus low enrolling site in SAMMPRIS (p=0.04, OR 3.9; Wald 95% CI 1.05 to 14.6). When high versus low enrolling site and subspecialty were included in a model along with the clinical factors that were associated with an increased risk of hemorrhagic stroke (per cent stenosis, modified Rankin score, and clopidogrel load associated with intraprocedural activated clotting time (ACTs) >300 s5), the p values were 0.058 for high versus low enrolling site and 0.70 for subspecialty.
In hemorrhagic subgroup analyses, the rates of SAH were 0% for neurosurgeons, 3.5% for radiologists, and 7.7%% for neurologists (p=0.08), 0.9% at high enrolling sites versus 4.9% at low enrolling sites in SAMMPRIS (p=0.11), and 0.9% among interventionists in practice for >8 years versus 4.8% among interventionists in practice <8 years (p=0.11) . There were no relationships between interventionist or site features and risk of ICH.
Ischemic infarcts (n=21)
Twenty-one ischemic infarcts (19 strokes, 2 CITS) occurred within 30 days of randomization in 213 patients undergoing PTAS. Of the 21 ischemic events, 15 occurred within 24 h of PTAS, five occurred between 24 h and 6 days after PTAS (of which two were definite or probable complete stent thromboses), and one (a CITS) occurred 3 weeks after PTAS. Fifteen of the 21 ischemic infarcts were categorized as involving a perforator territory. Table 3 shows the rates of any ischemic infarct and the subgroup with perforator infarct according to the various interventionist and site features that were evaluated. None of these features was significantly (p<0.05) associated with any ischemic infarct. The rates of any ischemic infarct were 12.6% at high enrolling sites in SAMMPRIS versus 6.9% at low enrolling sites (p=0.18).
Several important observations can be drawn from these data regarding the relationships between interventionist and site experience, and cerebrovascular complications after PTAS in the SAMMPRIS trial. First, the lack of a relationship between more extensive previous experience with the Wingspan stent (as evidenced by the number of Wingspan cases interventionists were credentialed with and whether a site was one of the highest enrolling sites in previous Wingspan registries) and a lower rate of periprocedural events in SAMMPRIS support our decision to allow self-expanding aneurysm stents and coronary stents for credentialing in SAMMPRIS. Interventionists credentialed for SAMMPRIS with fewer Wingspan cases typically made up their 20 cases with either a self-expanding aneurysm stent (Neuroform, Stryker, Kalamazoo, Michigan, USA), which has a similar delivery system to Wingspan, or balloon mounted coronary stents. The results of this study support the fact that good performance and outcome with these other stents were acceptable surrogates for good performance with the Wingspan stent. It should be noted that the credentialing data were self-reported: cases with bad outcomes may not have been included with the submitted operative notes and discharge summaries, and complications may have been under-recognized. However, the total number of cases performed with the device is likely to be reasonably accurate.
Second, the credentialing process in SAMMPRIS was effective as evidenced by the fact that interventionists who submitted fewer Wingspan cases for credentialing for the trial had a lower rate of ischemic events (p=0.05) and a similar rate of hemorrhagic stroke compared with investigators credentialed with more Wingspan cases (tables 2 and 3). These findings, coupled with the fact that the vast majority of interventionists credentialed for SAMMPRIS had been in practice for several years, argue strongly against the suggestion that operator inexperience with the device, or in general, was responsible for the higher than expected 30 day rate of stroke after PTAS in SAMMPRIS.7 While the association of less experience with the Wingspan stent prior to SAMMPRIS and a lower rate of cerebrovascular complications in SAMMPRIS seems paradoxical, it was not totally unexpected since the credentialing process was designed to include well trained, experienced, high quality interventionists even if they had not had extensive experience with Wingspan, as long as they had sufficient experience and good outcomes with another self-expanding stent or coronary stent.
Third, data from this study suggest a potential relationship between low enrollment and an increased risk for hemorrhagic stroke after PTAS. This was the only variable to reach statistical significance in bivariate analysis and it was nearly significant when included in a model that incorporated clinical factors associated with hemorrhage. We suspect that this association may, in part, be related to better familiarity and adherence to the PTAS protocol at high volume sites. Important components of the PTAS protocol to lower the risk of hemorrhagic stroke were maintaining an ACT between 250 and 300 s during the procedure and treating elevated blood pressure during and after the procedure with intravenous antihypertensive agents. While SAMMPRIS does show that high ACT levels associated with a loading dose of clopidogrel was an independent risk factor for periprocedural hemorrhagic stroke,5 we did not collect data on periprocedural blood pressures. As such, we cannot evaluate whether periprocedural blood pressure control was a contributing factor to the high rate of periprocedural hemorrhagic stroke in the trial.
On bivariate analyses, trends were seen between fewer years in practice and increased risk of SAH, and between physician specialty and increased risk of SAH and any hemorrhagic stroke. However, neither of these features was associated with hemorrhagic stroke in a multivariate analysis. Additionally, when clinical covariates associated with hemorrhagic stroke in SAMMPRIS5 were included in a multivariate analysis, the p values for subspeciality went from 0.07 in bivariate analysis to 0.70 in the multivariate analysis, indicating that the higher hemorrhagic event rate among neurology interventionists is probably explained by the clinical covariates—that is, the neurology interventionists treated patients with a higher prevalence of factors found to be related to the occurrence of a hemorrhage.
This study has important limitations: the analysis was post hoc, the number of interventionists was high, the number of cases done by each interventionist was low, and the number of 30 day events was low. As such, the likelihood of both type 1 error (random chance associations because of multiple comparisons) and type 2 errors (concluding no relationship exists when one in fact does but the study has insufficient power) is very high in this analysis. Also, the CIs for the difference between the percentages of patients with events show that with the small sample sizes we are unable to rule out that large differences may exist between groups for many of the factors. For these reasons, many of the findings in this analysis should be considered hypothesis generating.
Despite these limitations, the data from this analysis show unequivocally that: (1) the cerebrovascular complications from PTAS with the Wingspan system were widely distributed among many sites in the trial (ie, could not be explained by the poor performance of a few interventionists or sites); (2) the credentialing process in SAMMPRIS was effective in ensuring that interventionists credentialed with lower numbers of Wingspan procedures performed at least as effectively as interventionists credentialed with high numbers of Wingspan procedures; and (3) the poor outcome after PTAS compared with medical therapy alone in SAMMPRIS cannot be attributed to inexperience with the study device. Rather, the higher than expected rate of cerebrovascular complications in SAMMPRIS is more likely attributable to restricting inclusion in the trial to patients with severe (70–99%) stenosis and qualifying events within 30 days of enrollment as well as prospective and independent endpoint adjudication in SAMMPRIS. The original registries that reported much lower rates of periprocedural stroke included patients with 50–99% stenosis and patients with prior symptoms beyond 30 days. Both factors may be associated with lower procedural risks. Finally, patients in the registries were not prospectively and independently assessed for endpoint events and may have been under-reported as a consequence.
Given the unexpected and substantial decrease in the risk of stroke from aggressive medical therapy alone in SAMMPRIS, future endovascular approaches for this disease will need to focus on those subgroups in SAMMPRIS that had a high risk of stroke despite aggressive medical therapy. Perhaps patients with hemodynamic factors8 ,9 will turn out to be one of those subgroups. In addition, improvements in patient selection (eg, by imaging intracranial plaque using high resolution MRI10–12), reconsideration of less invasive endovascular approaches such as angioplasty alone,13 and improvements in devices will be necessary to substantially reduce the complication rate in order for endovascular therapy to be have a clearer role in the treatment of these patients.
Funding The SAMMPRIS trial was funded by a research grant (U01 NS058728) from the US Public Health Service National Institute of Neurological Disorders and Stroke (NINDS). In addition, the following Clinical and Translational Science Awards, funded by the National Institutes of Health, provided local support for the evaluation of patients in the trial: Medical University of South Carolina (UL1RR029882), University of Florida (UL1RR029889), University of Cincinnati (UL1RR029890), and University of California, San Francisco (UL1RR024131). Stryker Neurovascular (formerly Boston Scientific Neurovascular) provided study devices and supplemental funding for third party device distribution, site monitoring, and study auditing.
Competing interests CPD serves on the executive committee of the Stenting and Aggressive Medical Management for Preventing Recurrent stroke in Intracranial Stenosis (SAMMPRIS) trial which is funded by the National Institute of Neurological Disorders and Stroke (grant No U01 NS058728). He is a Co-PI on the SAMMPRIS trial and receives salary support from the SAMMPRIS grant. CPD also receives other grant support from the NINDS (P50 55977; R01 NS051631). He is also on the Scientific Advisory Board for W L Gore and Associates and is the Chair of the Scientific Advisory Board for Pulse Therapeutics. DF serves on the executive committee of the Stenting and Aggressive Medical Management for Preventing Recurrent stroke in Intracranial Stenosis (SAMMPRIS) trial which is funded by the National Institute of Neurological Disorders and Stroke (grant No U01 NS058728). He is a Co-PI on the SAMMPRIS trial and receives salary support from the SAMMPRIS grant. DF has received institutional research support from Siemens Medical and Microvention, consulting fees from Micrus-Johnson and Johnson, EV3/Covidian, Vascular Simulators, NFocus, W L Gore and Associates, and Microvention, and royalties from Micrus-Johnson and Johnson. MJL serves on the executive committee of the Stenting and Aggressive Medical Management for Preventing Recurrent stroke in Intracranial Stenosis (SAMMPRIS) trial which is funded by the National Institute of Neurological Disorders and Stroke (grant No U01 NS058728). He is PI of the SAMMPRIS Statistical Coordinating Center and receives salary support from the SAMMPRIS grant. MJL receives grant support from the National Eye Institute. He is the principal investigator of the Coordinating Center for Infant Aphakia Treatment Study (EY013287) and a co-investigator on the Core Grant for Vision Research (EY006360). SLB was an interventionalist on the Stenting and Aggressive Medical Management for Preventing Recurrent stroke in Intracranial Stenosis (SAMMPRIS) trial which is funded by the National Institute of Neurological Disorders and Stroke (grant No U01 NS058728). SLB has also been a consultant for Stryker Corporation. OOZ was an interventionalist on the Stenting and Aggressive Medical Management for Preventing Recurrent stroke in Intracranial Stenosis (SAMMPRIS) trial which is funded by the National Institute of Neurological Disorders and Stroke (grant No U01 NS058728). He has also acted as a consultant for EV3, Codman Neurovascular, Stryker Corporation and Microvention. PMM was an interventionalist on the Stenting and Aggressive Medical Management for Preventing Recurrent stroke in Intracranial Stenosis (SAMMPRIS) trial which is funded by the National Institute of Neurological Disorders and Stroke (grant No U01 NS058728). YPG was an interventionalist on the Stenting and Aggressive Medical Management for Preventing Recurrent stroke in Intracranial Stenosis (SAMMPRIS) trial which is funded by the National Institute of Neurological Disorders and Stroke (grant No U01 NS058728). JD was an interventionalist on the Stenting and Aggressive Medical Management for Preventing Recurrent stroke in Intracranial Stenosis (SAMMPRIS) trial which is funded by the National Institute of Neurological Disorders and Stroke (grant No U01 NS058728). BFL serves on the executive committee of the Stenting and Aggressive Medical Management for Preventing Recurrent stroke in Intracranial Stenosis (SAMMPRIS) trial which is funded by the National Institute of Neurological Disorders and Stroke (grant No U01 NS058728). BFL is the SAMMPRIS project manager and receives salary support from the grant. She has received consulting fees from Microvention Terumo. TNT serves on the executive committee of the Stenting and Aggressive Medical Management for Preventing Recurrent stroke in Intracranial Stenosis (SAMMPRIS) trial which is funded by the National Institute of Neurological Disorders and Stroke (grant No U01 NS058728). She is a Co-I on the SAMMPRIS trial. TNT is a past recipient of funding from the American Academy of Neurology (AAN) Foundation Clinical Research Training Fellowship and is the current recipient of a K23 grant from NIH/NINDS (1 K23 NS069668-01A1). LSJ is a program director at the National Institute of Neurological Disorders and Stroke. MIC is the grant recipient (U01 NS058728) for the NINDS funded clinical trial described in this paper. He has also received research grants from NINDS to fund the WASID trial (1 R01 NS36643) and to fund other research on intracranial stenosis (1 K24 NS050307 and 1 R01 NS051688). He currently serves on the stroke adjudication committee of an industry funded osteoporosis drug trial (Merck and Co Inc) and on the DSMB of another industry funded patent foramen ovale closure trial (W L Gore and Associates) and is compensated for those activities.
Provenance and peer review Not commissioned; not externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.