You are here

Article Text

Article menu

Download PDFPDF

SNIS 19th annual meeting electronic poster abstracts
E-107 Multi-center study of a deep learning model for intracranial aneurysm detection in computed tomography angiography
Free
  1. D Wu1,2,
  2. E Orru’3,
  3. S Ferraciolli4,
  4. A Ashok5,6,
  5. A Sorby-Adams7,8,
  6. M Guindy9,
  7. D Montes10,2,
  8. A Medina11,
  9. G Bianco4,
  10. J Kaggie5,6,
  11. D Rabina9,
  12. O Ashush12,
  13. X Han12,
  14. Y Baror12,
  15. M Patel12,
  16. I Dayan12,
  17. J Romero10,2,
  18. R González10,2,
  19. C Wald11,
  20. F Kitamura4,
  21. P Kuriki4,
  22. T Matys5,6,
  23. Q Li1,2
  1. 1Center for Advanced Medical Computing and Analysis, Massachusetts General Hospital, Boston, MA
  2. 2Harvard Medical School, Boston, MA
  3. 3Department of Interventional Neuroradiology, Lahey Hospital and Medical Center, Burlington, MA
  4. 4DasaInova, Diagnósticos da América SA (DASA), São Paulo, BRAZIL
  5. 5Department of Radiology, Addenbrooke’s Hospital, Cambridge University Hospital NHS Foundation Trust, Cambridge, UK
  6. 6Department of Radiology, University of Cambridge, Cambridge, UK
  7. 7MRC Mitochondrial Biology Unit, University of Cambridge, Cambridge, UK
  8. 8Department of Medicine, University of Cambridge, Cambridge, UK
  9. 9Department of Imaging, Assuta Medical Center, Tel Aviv-Yafo, ISRAEL
  10. 10Department of Radiology, Massachusetts General Hospital, Boston, MA
  11. 11Department of Radiology, Lahey Hospital and Medical Center, Burlington, MA
  12. 12Rhino HealthTech Inc., Boston, MA

Abstract

Introduction CT angiography (CTA), the modality of choice for detection of intracranial aneurysms (IA), is widely available at all care levels. Implementing deep learning algorithms (DLA) in the radiologists’ workflow could result in higher detection rates, particularly in smaller centers. We present results of multicenter pre-clinical experience with a new IA detection DLA.

Methods A DLA was developed at Site 1 to predict the existence and locations of IAs >2 mm in 3D head CTA volumes. It localizes IAs using an improved 3D UNet to regress the bounding boxes of suspicious locations. IA presence is confirmed by a multi-resolution 3D DenseNet at each suspicious location. The model was trained on 436 studies and tested on 696 studies from site 1. To validate robustness against protocol variability, it was retrospectively tested on 337 volumes (149 positives, 188 negatives) from 4 additional sites with various types of scanners. A neuroradiologist/trained fellow from each site annotated bounding boxes around the IAs with 3DSlicer. The model’s performance was assessed by comparing its predicted aneurysm locations to the annotated boxes.

Results The model achieved a lesion-level sensitivity of 0.72 (95% CI: 0.66 - 0.75) with 1.4 false positives per volume (FPPV, 95% CI: 1.22 - 1.57). Sensitivity ranged 0.84 (1.72 FPPV) - 0.67 (1.40 FPPV) amongst 5 sites. Performance variation was secondary to testing on images from scanners the model had not been trained on (unseen). Removing unseen scanners, the model achieved sensitivity of 0.77 (95% CI: 0.68 - 0.84) with 1.51 FPPV (95% CI: 1.24 - 1.82). Patient-level area under curves (AUCs) on the external sites were comparable to that of site 1 dataset, which was 0.88 (95% CI: 0.84 - 0.92). The model performed better in the detection of internal carotid and anterior communicating artery IAs > 3 mm.

View this table:
Abstract E-107 Table 1

Stratified performance of the model on the external datasets using the parameters set for Site 1

Abstract E-107 Figure 1
Abstract E-107 Figure 1

The free-response receiver operation characteristics (FROC) curve (left) and the ROC curve (right) of the model on the external (Site 2 – 5) and Site 1 datasets. The numbers in the FROC legends gave the average sensitivity at 0.125, 0.25, 0.5, 1, 2, 4, 8 FPPV. The numbers in the ROC legends gave the area under the curve (AUC)

Conclusions This DLA for IA detection showed robust performance in detecting IAs > 3 mm. DLAs may reliably help detection of IAs on CTA, particularly in the community practices less familiar with intracranial vascular imaging. Different scanners can lead to performance variability. This can be avoided by training with data from different scanners.

Disclosures D. Wu: 1; C; National Institute of Health. E. Orru’: None. S. Ferraciolli: None. A. Ashok: 1; C; NIHR Academic Clinical Fellowship. A. Sorby-Adams: None. M. Guindy: None. D. Montes: None. A. Medina: None. G. Bianco: None. J. Kaggie: 1; C; NIHR Cambridge BRC. D. Rabina: None. O. Ashush: None. X. Han: None. Y. Baror: None. M. Patel: 4; C; Rhino HealthTech Inc. I. Dayan: 4; C; Rhino HealthTech Inc.. J. Romero: None. R. González: 1; C; National Institute of Health. 2; C; MR Access, Inc.. 4; C; MR Access, Inc.. C. Wald: None. F. Kitamura: 2; C; MD.ai, GE Healthcare. P. Kuriki: None. T. Matys: 1; C; NIHR Cambridge BRC. 2; C; NHS. Q. Li: 1; C; National Institute of Health.

http://dx.doi.org/10.1136/neurintsurg-2022-SNIS.218

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.