Article Text

Original research
Endovascular thrombectomy time metrics in the era of COVID-19: observations from the Society of Vascular and Interventional Neurology Multicenter Collaboration
  1. Alexandra L Czap1,
  2. Alicia M Zha1,
  3. Jacob Sebaugh1,
  4. Ameer E Hassan2,3,
  5. Julie G Shulman4,
  6. Mohamad Abdalkader4,
  7. Thanh N Nguyen4,5,6,
  8. Italo Linfante7,
  9. Amy Kathryn Starosciak8,
  10. Santiago Ortega-Gutierrez9,
  11. Mudassir Farooqui9,
  12. Darko Quispe-Orozco9,
  13. Nirav A Vora10,
  14. Vivek Rai10,
  15. Raul G Nogueira11,
  16. Diogo C Haussen11,
  17. Dinesh V Jillella12,
  18. Ameena Rana13,
  19. Siyuan Yu13,
  20. Jesse M Thon14,
  21. Osama O Zaidat15,
  22. Priyank Khandelwal16,
  23. Ivo Bach16,
  24. Sunil A Sheth1,
  25. Ashutosh P Jadhav17,18,
  26. Shashvat M Desai17,18,
  27. Tudor G Jovin14,
  28. David S Liebeskind19,
  29. James E Siegler14
  1. 1 Neurology, McGovern Medical School at the University of Texas Health Science Center, Houston, Texas, USA
  2. 2 Department of Neurology, University of Texas Rio Grande Valley, Harlingen, Texas, USA
  3. 3 Clinical Neuroscience Research, Valley Baptist Medical Center, Harlingen, Texas, USA
  4. 4 Neurology, Boston Medical Center, Boston, Massachusetts, USA
  5. 5 Radiology, Boston Medical Center, Boston, Massachusetts, USA
  6. 6 Interventional Neuroradiology and Endovascular Neurosurgery, Boston Medical Center, Boston, Massachusetts, USA
  7. 7 Department of Interventional Neuroradiology and Endovascular Neurosurgery, Baptist Health South Florida, Coral Gables, Florida, USA
  8. 8 Center for Outcomes Research, Baptist Health South Florida, Coral Gables, Florida, USA
  9. 9 Neurology, Neurosurgery and Radiology, The University of Iowa Hospitals and Clinics, Iowa City, Iowa, USA
  10. 10 Neuroscience Center, Riverside Methodist Hospital, Columbus, Ohio, USA
  11. 11 Neurology, Grady Memorial Hospital, Atlanta, Georgia, USA
  12. 12 Neurology, Emory University, Atlanta, Georgia, USA
  13. 13 Cooper Medical School of Rowan University, Camden, New Jersey, USA
  14. 14 Cooper Neurological Institute, Cooper University Health Care, Camden, New Jersey, USA
  15. 15 Neuroscience Institute, Bons Secours Mercy Health St. Vincent Hospital, Toledo, Ohio, USA
  16. 16 Neurology, Rutgers New Jersey Medical School, Newark, New Jersey, USA
  17. 17 Neurology, University of Pittsburgh Medical Center Mercy Hospital, Pittsburgh, Pennsylvania, USA
  18. 18 Neurology, University of Pittsburgh Medical Center Presbyterian Medical Center, Pittsburgh, Pennsylvania, USA
  19. 19 Neurology, Ronald Reagan UCLA Medical Center, Los Angeles, California, USA
  1. Correspondence to Dr James E Siegler, Cooper Neurological Institute, Cooper University Health Care, Camden, NJ 08103-1489, USA; siegler.james{at}


Background Unprecedented workflow shifts during the coronavirus disease 2019 (COVID-19) pandemic have contributed to delays in acute care delivery, but whether it adversely affected endovascular thrombectomy metrics in acute large vessel occlusion (LVO) is unknown.

Methods We performed a retrospective review of observational data from 14 comprehensive stroke centers in nine US states with acute LVO. EVT metrics were compared between March to July 2019 against March to July 2020 (primary analysis), and between state-specific pre-peak and peak COVID-19 months (secondary analysis), with multivariable adjustment.

Results Of the 1364 patients included in the primary analysis (51% female, median NIHSS 14 [IQR 7–21], and 74% of whom underwent EVT), there was no difference in the primary outcome of door-to-puncture (DTP) time between the 2019 control period and the COVID-19 period (median 71 vs 67 min, P=0.10). After adjustment for variables associated with faster DTP, and clustering by site, there remained a trend toward shorter DTP during the pandemic (βadj=-73.2, 95% CI −153.8–7.4, Pp=0.07). There was no difference in DTP times according to local COVID-19 peaks vs pre-peak months in unadjusted or adjusted multivariable regression (βadj=-3.85, 95% CI −36.9–29.2, P=0.80). In this final multivariable model (secondary analysis), faster DTP times were significantly associated with transfer from an outside institution (βadj=-46.44, 95% CI −62.8 to – -30.0, P<0.01) and higher NIHSS (βadj=-2.15, 95% CI −4.2to – -0.1, P=0.05).

Conclusions In this multi-center study, there was no delay in EVT among patients treated for intracranial occlusion during the COVID-19 era compared with the pre-COVID era.

  • stroke
  • thrombectomy
  • COVID-19

Data availability statement

Data are available upon reasonable request. The data that support the findings of this study are available from the corresponding author upon reasonable request, after clearance by the local ethics committee.

This article is made freely available for use in accordance with BMJ’s website terms and conditions for the duration of the covid-19 pandemic or until otherwise determined by BMJ. You may use, download and print the article for any lawful, non-commercial purpose (including text and data mining) provided that all copyright notices and trade marks are retained.

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.


Coronavirus disease 2019 (COVID-19) has placed unprecedented stress on the US healthcare system. Delays in the evaluation of acute stroke and treatment with intravenous thrombolytics during the COVID-19 pandemic have been previously demonstrated.1–3 However, little has been reported in regard to patients harboring intracranial occlusions who undergo endovascular thrombectomy (EVT).4 5 Guidelines during the COVID-19 pandemic have called for streamlined and efficient processes,6–11 however, there remain considerable variations in management paradigms with ongoing debates, including recommendations for pre-thrombectomy airway management, intravenous thrombolysis, and antithrombotics.12 Due to the enforcement of contact precautions, the use of personal protective equipment, and decontamination procedures for imaging and treatment facilities, it stands to reason that the COVID-19 pandemic may result in delays of acute interventions for stroke. Furthermore, delays in acute care may also be affected by local surges in COVID-19 cases as healthcare systems become overrun and resources exhausted.13 14]

The aim of the present study was to evaluate the impact of COVID-19 on the emergent management of acute intracranial occlusions. We hypothesized that the COVID-19 pandemic would lead to a delay in EVT time metrics, similar to the delay we have shown among patients treated with intravenous thrombolysis.1


The data that support the findings of this study are available from the corresponding author on reasonable request, after clearance by the local ethics committee.

Study population

We conducted a retrospective review of our prospectively maintained observational registry data to identify all consecutive patients with acute ischemic stroke and intracranial occlusion who underwent emergent angiography and/or thrombectomy at 14 Comprehensive Stroke Centers (CSCs) across nine US states between January 1, 2019 and July 31, 2020. At the time of data consolidation, these states accounted for 47% of all reported US cases of COVID-19 and 37% of COVID-19-associated deaths.15 Participating registry centers were recruited based on their affiliation with the Society of Vascular and Interventional Neurology.

Patients were eligible for inclusion if they experienced an acute intracranial occlusion of the internal carotid artery (ICA), middle cerebral artery (inclusive of M1, M2, or M3 segments), or vertebral and/or basilar arteries. Occlusion location was determined by CT angiography or by digital subtraction angiography. Patients were excluded if they were triaged by a mobile stroke unit (with these data being reported separately), if the stroke occurred while hospitalized (in-hospital stroke) given unique triage protocols associated with patient populations, and if they were admitted outside the study periods as outlined below.

Study periods

To maximize sensitivity for detecting differences in treatment times based on the differential burden and delayed spread of the novel coronavirus among the included sites, two analyses were performed. In the primary analysis, patients admitted between March 1, 2019 and July 31, 2019 (seasonal control period) were compared against patients admitted between March 1, 2020 and July 21, 2020 (COVID-19 period) across all sites. A secondary analysis was conducted based on relative COVID-19 surges affecting sites on the state level, according to data published by the New York Times.16 In this analysis, patients were compared between two contemporaneous months around the time of the first local COVID-19 “peak”. Months were selected for state-specific COVID-19 peak comparisons using a modified Delphi consensus between first, second, and senior authors (ALC, AMZ, JES) using qualitative visualization of new daily COVID-19 diagnoses and total COVID-19 hospitalizations in that state.16 In this analysis, the month of the first clinically significant increase in daily COVID-19 diagnoses and/or hospitalizations was categorized as the COVID-19 peak. To maximize sensitivity of the state-specific COVID-19 peak comparisons, the pre-peak (referent) period preceded the COVID-19 peak period by 1 month. For example, if the COVID-19 peak occurred in May 2020, the referent month would be March 2020 for that state. In this analysis, only the time of the first peak was used and only months through July 2020 could be selected as COVID-19 peak months due to the recent nature of multicenter data consolidation. For states in which new daily COVID-19 diagnoses did not have a clear peak (eg, Iowa), the peak of COVID-19 hospitalizations was used as the primary indicator of COVID-19 peak.

Variables and outcomes

Baseline demographic data including age (which was binned by decade to protect personal health information), sex, race, and ethnicity, as well as key vascular risk factors were collected from participating sites. Patient arrival method (use of emergency medical services (EMS) vs walk-in/private vehicle vs transfer from outside hospital), baseline stroke severity according to the National Institutes of Health Stroke Scale (NIHSS), occlusion location, and treatment with intravenous thrombolysis and/or EVT are also reported. To account for site EVT volume as a confounder, we included a variable recognizing sites as ‘high-volume’ if the 2019 EVT volume exceeded 50, in accordance with previously published data indicating faster treatment times.1 17

The primary outcome was door-to-puncture time, assessed as a continuous variable. We further explored the time windows from last known well (LKW) to arrival at the CSC, door to initial imaging, puncture to first pass, puncture to recanalization (or completion of angiography if recanalization unsuccessful), and overall LKW to recanalization. Additional endpoints included successful recanalization (defined as a thrombolysis in cerebral infarction (TICI) score of 2b/3) and discharge disposition (stratified by home, acute inpatient rehabilitation, other facility, or hospice/death which was consolidated). Due to the incompleteness of data regarding discharge mRS scores (48% completion rate) and lack of available 90-day mRS scores for recently discharged patients from the COVID-19 pandemic, functional and long-term outcomes are not reported.

Statistical analysis

Categorical variables are reported using absolute counts and proportions, while non-normally distributed continuous variables are reported using medians and interquartile ranges (IQRs). Baseline characteristics between pre-COVID-19 and COVID-19 groups were compared using the Chi-square test or Fisher’s exact test for categorical variables, when appropriate, and the Wilcoxon rank-sum test for continuous variables. Unadjusted logistic and linear regression models were generated to estimate the effect of the COVID-19 period on the primary and secondary outcomes of interest. An adjusted linear regression model was used to estimate the impact of the COVID-19 period on any potential delay in the primary outcome of door-to-puncture time, with adjustment for all variables significant to P≤0.1 in univariate regression. Each adjusted regression model was clustered by site.

All statistical tests were 2-sided and conventional levels of significance (α=0.05) were used for interpretation, with P-values reported for conventional purposes only. Odds ratios and beta coefficients were reported with 95% confidence intervals (OR or β [95% CI]). No adjustments were made for multiple comparisons as analyses were exploratory and hypothesis-generating. Missing data were not imputed. STATA 15.0 (StataCorp, LLC) software was used for data analysis.

Ethics statement

The local institutional review board (Committee for the Protection of Human Subjects) at each participating center reviewed and approved the study with waiver of consent and HIPAA authorization was granted. This study is reported in accordance with the Strengthening the Reporting of Observational Studies in Epidemiology statement.


Primary analysis

Of 12 187 patients screened from the multicenter registry, 6277 patients were admitted within the study window for the primary analysis. Of these, 4868 were excluded due to lack of intracranial occlusion, 26 for experiencing an in-hospital stroke, and 18 due to initial evaluation in a mobile stroke unit, leaving 1364 included in the primary analysis (online supplemental figure 1). More than half (51%) were ≥70 years of age, 48% were female, and 61% were White (table 1). The median NIHSS was 14 (IQR 7–21), and 74% had an occlusion of the internal carotid, proximal middle cerebral (M1), or basilar artery. There was a 28% decrease in the total number of patients with any intracranial occlusion during the COVID-19 period when compared with the 2019 seasonal control period (570 vs 794 patients). Compared with patients who presented in the seasonal control period, patients who presented in the COVID-19 period were more frequently Hispanic (OR 1.40, 95% CI 1.03 to 1.89, P=0.03), had coronary artery disease (OR 1.91, 95% CI 1.41 to 2.58, P<0.01), and had a higher probability of undergoing EVT (OR 1.39, 95% CI 1.08 to 1.78, P=0.01).

Table 1

Demographics and clinical characteristics by seasonal period

For the primary outcome of door-to-puncture time, there was no significant difference in door-to-puncture times among patients admitted during the COVID-19 period vs the seasonal control period (β=−41.9, 95% CI −93.0–9.1, P=0.11; see table 2 for absolute time differences). After adjustment for all candidate variables associated with door-to-puncture (P<0.1: history of hypertension, diabetes, dyslipidemia, transfer from outside institution, and baseline NIHSS) and clustering by site, there was a trend toward faster door-to-puncture times during the COVID-19 period (βadj=-73.2, 95% CI −153.8–7.4, P=0.07; table 3). While transfer from an outside institution was significantly associated with faster door-to-puncture time in univariate regression (P=0.04), it was no longer associated with faster treatment in multivariable regression (padj=0.30). Only a history of hypertension was independently associated with delayed EVT during the COVID-19 period (βadj=57.1, 95% CI −1.3–115.5, P=0.05). A greater proportion of patients were discharged home over acute inpatient rehabilitation during the COVID-19 period when compared with the control period (38% vs 33%, P<0.01; table 2).

Table 2

Procedural and clinical outcomes of EVT by seasonal period

Table 3

Candidate variables associated with door-to-puncture time in the primary analysis

Secondary analysis

For the secondary analysis of state-specific COVID-19 peaks, there was 100% agreement among the three raters regarding each state’s first COVID-19 peak (figure 1). In this analysis, 123 patients were admitted in the state-specific pre-peak month and 84 patients were admitted during the state-specific COVID-19 peak (32% decline during local COVID-19 peak). Compared with the pre-peak month, patients admitted during the local COVID-19 peak more frequently had atrial fibrillation (OR 2.18, 95% CI 1.12 to 4.26, P=0.02) with a trend toward lower odds of prior stroke (OR 0.43, 95% CI 0.18 to 1.02, P=0.06). There was a higher prevalence of basilar occlusion during the COVID-19 peak (OR 5.41, 95% CI 1.44 to 20.28, P=0.01), but no other differences with respect to age, sex, or comorbidities (online supplemental table 1).

Figure 1

State-specific COVID-19 peak and pre-peak periods for secondary analysis. April 2020 was selected as the month of the first COVID-19 peak for New Jersey (n=2 sites), Pennsylvania (n=2 sites), and Massachusetts (n=1 site); May 2020 as the first COVID-19 peak for Iowa (n=1 site); and July 2020 the first COVID-19 peak for Texas (n=2 sites), Ohio (n=2 sites), Georgia (n=2 sites), Florida (n=1 site), and California (n=1 site). COVID-19 denotes coronavirus disease 2019.

For the primary outcome of door-to-puncture, there was no significant difference in treatment times among patients admitted during the local COVID-19 peak vs the pre-peak month (β=2.29, 95% CI −23.9–28.5, P=0.86) and there remained no significant difference after adjustment for all candidate variables included in the multivariable regression model (P<0.1: White race, transfer from an outside institution, NIHSS, and occlusion of the ICA/M1/basilar artery), and after clustering by site (βadj=-3.85, 95% CI −29.9–22.2, P=0.77; online supplemental table 2). In this final multivariable model, faster door-to-puncture times were significantly associated with transfer from an outside institution (βadj=-46.44, 95% CI −62.8 to−30.0, P<0.01) and higher NIHSS (βadj=-2.15, 95% CI −4.2 to−0.1, P=0.05).


In this multicenter investigation evaluating treatment times of 1364 patients with an acute intracranial occlusion across nine US states, we observed no significant delay in thrombectomy treatment time during the COVID-19 pandemic. Our findings of a nearly one-third reduction in LVO incidence may be in part due to a reduction in patients seeking care during the pandemic,18 rather than a reduction in LVO stroke incidence. The increased likelihood of EVT is not fully explained in the present analysis, and it may be related to patient-level factors and imaging findings (eg, estimated infarct volume, collateral status, perfusion findings) which were not collected as part of this observational study. Declines in LVO rate during the COVID-19 period (and in the state-specific COVID-19 peaks) were not driven by any one particular center, as 11 sites observed the same effect (three sites had similar or higher case rates during COVID-19; data not otherwise shown). Furthermore, we observed a trend toward shorter door-to-puncture times during the COVID-19 period when compared with the 2019 seasonal control period. This may be related, at least in part, to the lower volume of elective angiographic and neurointerventional cases as part of the healthcare response to the pandemic,19 20 as well as the lower volume of emergent endovascular cases.21 The shorter time between arrival and angiography may also be related to the manner in which patients arrived to CSCs included in this analysis. More than half of patients were transferred from an outside institution (60% of cohort), and these individuals were generally treated with greater expediency when compared with non-transferred patients. Longer door-to-puncture delays during the COVID-19 period were observed among included patients who arrived at the CSC using private transportation, which is unsurprising given prior experiences.22 23 Pre-hospital evaluation and notification by EMS and external providers can expedite the preparedness of the CSC interventional team and shorten delays from CSC arrival to skin puncture.24 25 While only a small proportion of patients included in this analysis presented by private vehicle/walk-in (7% of cohort), the effect of private transport on EVT throughput is likely a significant underestimation of the national (and international) experience. This study was limited to the experience of CSCs which are vastly outnumbered by Primary Stroke Centers, urgent care facilities, and community hospitals where these patients may initially present. Therefore, we would continue to encourage patients to call for emergency services in the event of a sudden, significant neurologic symptom. Earlier symptom recognition in the field and at non-CSCs can expedite reperfusion therapies.

Regarding other time metrics in the acute care of patients with intracranial occlusion, we found no significant delay in arrival to first head imaging at participating CSCs. Presumably, any delay in the acute care of patients treated in the emergency department during the COVID-19 pandemic would relate to initial triage as providers are committing more time and attention to contact precautions, hygiene, and decontamination. However, there appeared to be no difference in time from CSC arrival to first head imaging between patients admitted during COVID-19 vs the 2019 seasonal control period (or among patients admitted during local COVID-19 peaks vs the preceding period). In a related study, we found a small but significant 8-min decrease in arrival to first head imaging among patients evaluated during the COVID-19 pandemic, including patients without an intracranial occlusion (median 29 vs 37 min, P<0.01).1 It is possible that faster imaging times may not have been observed in this cohort due to its smaller sample size or more restrictive inclusion criteria.

Unsurprisingly, patients who were transferred from an outside hospital with an acute ischemic stroke had faster treatment times. In the primary analysis, shorter door-to-puncture times among patients transferred vs those who arrived initially at the CSC was no longer significant after adjustment for measurable confounders. However, when state-specific COVID-19 peaks were evaluated, there remained a strong and independent relationship between transfer status and shorter door-to-puncture times.

Of note, there was a significant difference in discharge disposition among hospitalized patients during COVID-19 when compared with the seasonal control. This is likely mediated by several indirect effects of the pandemic. First, patients admitted during COVID-19 typically have more severe disease and more comorbidities, as we have shown in an overlapping cohort.26 Therefore, the in-hospital mortality risk would be expectedly higher in this population. Second, unprecedented depletion of acute care beds to manage patients with COVID-19 has been met with a commensurate decrease in available inpatient rehabilitation and nursing beds for discharge-eligible patients.27 In order to safely and efficiently discharge patients with stroke and other medical diseases, many patients have been (and are being) discharged to home with home or outpatient services, and close neuromedical follow-up. It is possible that a small proportion of these patients would have otherwise benefited from more aggressive inpatient rehabilitation, however constraints on medical infrastructure have demanded creative, patient-level responses during this pandemic.28 29

Our investigation is unique in that we conducted an analysis that accounted for the first local COVID-19 peak in each state. As shown in figure 1, several states (notably California, Florida, Georgia, Ohio, and Texas) experienced delayed surges in COVID-19 cases. Therefore, an analysis limited to the first few months of the pandemic in the US might have been too early to detect any impact of COVID-19 surges on treatment times in these regions. In this secondary analysis–while the sample size was lower than that of the primary analysis–there was still no statistically or clinically significant difference in treatment times between pre-peak and peak periods.


While this study is the largest pooled data set reflecting multiple unique regions in the US affected by the COVID-19 pandemic, it is limited by its retrospective and exploratory nature. The brief study periods and small sample size included in the primary and secondary analyses may have contributed to a type II error. Certain data elements, such as vital signs on admission and variables related to social history or emergency department volume, may have also contributed to delays in endovascular treatment times, but these data were not available from the consolidated prospective registries. In spite of these limitations, we still believe there to be no clinically significant delay in EVT as a consequence of the pandemic. If anything, there appeared to be a trend toward shorter arrival to EVT times during the COVID-19 pandemic when compared with identical months in the preceding year (padj=0.07).

The lack of available long-term outcome data in the present analysis, and lack of adjustment for COVID-19 infection status among included patients are other shortcomings of this study. However, as we have previously shown using data that overlaps with this registry,30 the prevalence of ischemic stroke in patients with COVID-19 was 1.1%–1.4% with approximately half of these patients harboring an intracranial occlusion. Although COVID-19-associated stroke is typically more severe, this low prevalence is unlikely to significantly confound the results of this investigation. Due to the recency of data consolidation for this analysis, long-term follow-up data remain unavailable. That said, the impact of delays in door-to-puncture have been extensively described in relation to long-term functional outcomes.31

In this multi-center observational registry representing the experience at 14 CSCs in nine US states, which accounted for nearly half of all reported COVID-19 cases during the early wave of the pandemic, we found no significant delay in thrombectomy among patients with acute intracranial occlusion. While the number of new LVO cases fell considerably, the proportion of thrombectomies increased among all intracranial occlusions, and among patients with proximal occlusions. While patients with mild strokes may be avoidant of the hospital,26 further research is called upon to explore potential reasons for the decline in new LVOs. The majority of patients in this analysis were transferred from an outside institution to the CSC, and in general, treatment times were quicker for these patients when compared with patients who presented via EMS or private vehicle. Treatment times remained significantly and independently shorter among patients who were transferred in the secondary analysis evaluating local COVID-19 peaks. Slower treatment among patients who present directly to CSC emergency departments remains a potential gap for the improvement in the care of patients with acute stroke. Centers are encouraged to evaluate their local paradigms for opportunities to facilitate the throughput of patients who arrive directly to their emergency department with stroke due to intracranial occlusion.

Data availability statement

Data are available upon reasonable request. The data that support the findings of this study are available from the corresponding author upon reasonable request, after clearance by the local ethics committee.

Ethics statements

Patient consent for publication


Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.


  • Twitter @AlexandraCzap, @tele_Zha, @italolinfante, @NiravAVora, @diogohaussen, @dineshjillella, @ashupjadhav, @shashvatdesai, @JimSiegler

  • ALC and JES contributed equally.

  • Contributors AC and JS were responsible for the drafting and critical review of this manuscript. JS performed the statistical analyses. All authors were responsible for data collection at their respective sites. All authors have reviewed the manuscript, provided editorial feedback, and approve of its submission.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial, or not-for-profit sectors.

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.