Article Text

Download PDFPDF

Lies, damned lies, and TICI
  1. Maxim Mokin1,
  2. Joshua A Hirsch2,
  3. David Fiorella3,
  4. Ashutosh P Jadhav4
  1. 1 Neurosurgery, University of South Florida, Tampa, Florida, USA
  2. 2 NeuroEndovascular Program, Massachusetts General Hospital, Boston, Massachusetts, USA
  3. 3 Department of Neurosurgery, Stony Brook University, Stony Brook, New York, USA
  4. 4 Neurology, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
  1. Correspondence to Dr Maxim Mokin, Department of Neurosurgery, University of South Florida, 2 Tampa General Circle, Tampa, FL 33606, USA; mokin{at}usf.edu

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

“Failure is success in progress,” is one of Albert Einstein’s quotes, emphasizing the importance of recognizing one’s own limitations. The devices we use in our daily neurointerventional practice are constantly undergoing changes and improvements, as well as the techniques and approaches we apply to treat a wide range of cerebrovascular pathologies. Yet, we continue to judge procedural success or failure based on the visual feedback—the approach that has remained fundamentally the same since the introduction of the digital subtraction angiography technique.

The Thrombolysis in Cerebral Infarction (TICI) scoring system is the uniform 'currency' we all use every day; it tells us when the thrombectomy procedure is completed or whether additional device passes are required, it is used by the regulatory agencies to evaluate device safety, and it allows us to compare trial results and determine if the 'next best thing' truly achieves superior and faster reperfusion rates, as claimed by the authors.1 It is not surprising therefore that the TICI scale has quickly found its way in the social media arena, with endless posts announcing first pass TICI 3 results greatly surpassing the clinical trial data or with claims of new devices’ 'superiority'. Dmytriw et al compared post-intervention Twitter reported thrombectomy outcomes with the Highly Effective Reperfusion Evaluated in Multiple Endovascular Stroke (HERMES) individual patient meta-analysis and showed a striking (over 20%) difference in the reported rates of modified TICI (mTICI) score of 2c/3 (94% vs 71%, respectively; p<0.0001).2

Successful reperfusion, which is traditionally defined as mTICI 2b/3 recanalization, is an extremely powerful predictor of a stroke patient becoming functionally independent in the future. Given its importance, here we aimed to review the data on the reliability and reproducibility of this angiographic scale, and to discuss its limitations and possible implications in patient care. Inter-rater or inter-observer reliability describes the accuracy of an individual’s observation of a phenomenon, which in our case will be the rater’s ability to accurately and consistently assign TICI ranges. It is a well described phenomenon that individual operators tend to overcall the degree of procedural success. Core laboratory assessments have thus become the gold standard in evaluating angiography in many interventional disciplines, including neuro-interventions.

How frequently this discrepancy occurs is elegantly illustrated in the MR CLEAN (Multicenter Randomized Clinical Trial of Endovascular Treatment for Acute Ischemic Stroke in the Netherlands) registry study where operator mTICI and core laboratory mTICI scores were compared.3 In data derived from 1130 patients with anterior circulation large vessel occlusion treated with thrombectomy, the proportion of concordance between operator and core laboratory mTICI score was 56% (95% CI 54% to 59%). Furthermore, in every 1 in 3 cases (33% of cases), mTICI was overestimated by operators (95% CI 31% to 36%). Similarly, in the ASTER (Contact Aspiration vs Stent Retriever for Successful Revascularization) trial, disagreements in mTICI scores between study sites and the core laboratory were substantial (29.7% and 25.4% of cases for intermediate and final scores, respectively).4

The TICI scale was originally designed for evaluating territory reperfusion in the middle cerebral artery, and is currently used for other arterial territories, including the posterior circulation.5 In a recent study of eight observers asked to evaluate TICI scores before and after thrombectomy of posterior circulation occlusions, Findler et al showed high variability of individual scores (k value reached 0.277) corresponding to a 'fair' agreement among the raters.6 The authors specifically noted low inter-rater reliability of attributes such as occlusion severity, recanalization status, endangered territories, and the existence of basilar atherosclerotic disease, questioning the utility of TICI in this population of patients.

What can we learn from these studies? Relying entirely on core laboratory adjudication of individual studies would help increase the accuracy of angiographic data, but it is not practical. One solution is to have an independent operator from the same institution review the results independently, rather than relying on records of dictated reports. However, neither core laboratory nor individual operator adjudication would apply to the most critical scenario, that is, when the treating neurointerventionist is in the angiography suite performing the thrombectomy procedure and needs to decide in a timely manner whether additional device passes or therapeutic interventions are needed. Individual experience, fatigue and stress, quality of image processing, and other potential biases likely all contribute to the operator’s ability to grade the TICI result accurately.

Awareness of the limitations of TICI is a good start. One of the authors of this article recently performed a 'quality check' by going back and reviewing angiographic outcomes of prior thrombectomy procedures, which demonstrated overestimation and underestimation of individual mTICI results by 20% and 8%, respectively, especially in cases requiring multiple passes and those occurring during off-hours. This was an important reality check, highlighting the need for a more thorough evaluation of angiographic results in future cases. Table 1 lists potential solutions to improve the subjectivity of angiographic assessment of thrombectomy. Ongoing research is evaluating whether alternative angiographic scales, such as the expanded TICI (eTICI) score, could further improve the operator’s accuracy of determining their procedural success.7 Finally, technological advances, such as incorporating real time automated TICI software into stroke thrombectomy procedures, may serve as an additional safeguard in helping physicians recognize hyperperfused areas.8 9 All of these are works in progress and creating awareness of this matter is the first step in the right direction.

Table 1

Solutions to improve operator’s accuracy of angiographic adjudication of reperfusion during stroke thrombectomy

More studies similar to the MR CLEAN registry analysis are needed to confirm how prevalent the issue of TICI overestimation is depending on stroke severity, location of occlusion, neurointerventional team set up, and operator experience. As more data become available, establishing a core laboratory correction factor could be considered when comparing and interpreting the results among individual operators, institution data, or retrospective data with the TICI results reported in thrombectomy trials with independent core laboratory adjudications. Such an approach was previously successfully applied by Fiorella et al when comparing angiographic occlusion scales of wide necked bifurcation aneurysms.10 To do so, the authors relied on three separate studies where both core laboratory and individual operator estimates of aneurysm embolization after coiling was reported. This wide necked bifurcation aneurysm correction factor is already used by regulatory agencies such as the Food and Drug Investigation as Objective Performance Criteria (OPC) for the evaluation of novel devices for the treatment of wide necked bifurcation aneurysms. The challenges of TICI self-adjudication mandate the need for a similar core laboratory correction factor based on multiple high quality data sets.

Ethics statements

References

Footnotes

  • Twitter @JoshuaAHirsch

  • Contributors All authors contributed equally.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial, or not-for-profit sectors.

  • Competing interests None declared.

  • Provenance and peer review Commissioned; internally peer reviewed.