Responses

Download PDFPDF
Original research
Using machine learning to optimize selection of elderly patients for endovascular thrombectomy
Compose Response

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
Author Information
First or given name, e.g. 'Peter'.
Your last, or family, name, e.g. 'MacMoody'.
Your email address, e.g. higgs-boson@gmail.com
Your role and/or occupation, e.g. 'Orthopedic Surgeon'.
Your organization or institution (if applicable), e.g. 'Royal Free Hospital'.
Statement of Competing Interests

PLEASE NOTE:

  • A rapid response is a moderated but not peer reviewed online response to a published article in a BMJ journal; it will not receive a DOI and will not be indexed unless it is also republished as a Letter, Correspondence or as other content. Find out more about rapid responses.
  • We intend to post all responses which are approved by the Editor, within 14 days (BMJ Journals) or 24 hours (The BMJ), however timeframes cannot be guaranteed. Responses must comply with our requirements and should contribute substantially to the topic, but it is at our absolute discretion whether we publish a response, and we reserve the right to edit or remove responses before and after publication and also republish some or all in other BMJ publications, including third party local editions in other countries and languages
  • Our requirements are stated in our rapid response terms and conditions and must be read. These include ensuring that: i) you do not include any illustrative content including tables and graphs, ii) you do not include any information that includes specifics about any patients,iii) you do not include any original data, unless it has already been published in a peer reviewed journal and you have included a reference, iv) your response is lawful, not defamatory, original and accurate, v) you declare any competing interests, vi) you understand that your name and other personal details set out in our rapid response terms and conditions will be published with any responses we publish and vii) you understand that once a response is published, we may continue to publish your response and/or edit or remove it in the future.
  • By submitting this rapid response you are agreeing to our terms and conditions for rapid responses and understand that your personal data will be processed in accordance with those terms and our privacy notice.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.

Vertical Tabs

Other responses

Jump to comment:

  • Published on:
    SPOT as a clinical decision aid
    • Ali Alawieh, Graduate Researcher Department of Neurosurgery, Medical University of South Carolina
    • Other Contributors:
      • Fadi Zaraket, Associate Professor in Electrical and Computer Engineering
      • Mohamed Baker Alawieh, Graduate Assistant
      • Alejandro Spiotta, Neurosurgeon and Neuroendovascular surgeon

    We read with interest the response to our manuscript on using machine learning to optimize elderly patient selection for endovascular thrombectomy (1). We acknowledge here, as the author reports, the limitation of SPOT being based on single center data, and the need for multicenter prospective validation of SPOT as next step in development. The author raises additional technical concerns that we do not necessarily view as applicable to this study.

    First, we would like to stress the general limitations of artificial intelligence based techniques such as the overfitting and the data specific local optima problems. However, the specific comments brought by the author are not applicable in our case. First, studies on the number of events per predictor are applicable for logistic regressions (LRs) which is not used in the SPOT algorithm. In fact, our results show poor LR performance which is consistent with the rule of thumb of 1 to 10 referred to by the author. Hence, while serving as a good guidance for LR, the rule is not binding and more importantly it does not guarantee the generalization of the learned model. To further illustrate, classification models using convolutional neural networks have millions of parameters and are trained with datasets that, in most cases, do not have millions of samples in each group. However, these models have acceptable generalization capabilities and are tested using the data-split method. In SPOT, the model at its core is a regressi...

    Show More
    Conflict of Interest:
    None declared.
  • Published on:
    Response to: "Using machine learning to optimize selection of elderly patients for endovascular thrombectomy"
    • F.C.C. Kremers, Research master student Erasmus University Medical Center Rotterdam
    • Other Contributors:
      • E. Venema, PhD Candidate
      • H.F. Lingsma, Statistician
      • D.W.J. Dippel, Neurologist

    It is with great interest that we read the study of Alawieh et al(1), in which they developed a machine learning algorithm, called ‘SPOT’, to select stroke patients older than 80 years for endovascular therapy (EVT). Prediction modeling to optimize patient selection for EVT is an emerging topic of interest and we agree that predicting individual patient outcomes is increasingly important for decision making in medicine. However, we were surprised by the strong conclusions that were drawn by the authors, considering some serious limitations of the study.

    First, the size of the training set is insufficient to develop a complex model with twelve predictor variables and many correlations. Only 22 patients had a good functional outcome, which means that the number of events per tested predictor variable is less than two. For the development of a reliable model, a sample size of at least ten events per variable is needed to minimize the risk of overfitting(2, 3). It has been suggested that even far more events per variable are needed to achieve stable predictions with machine learning techniques(4). Especially complex models developed on small sample sizes have a high risk of overfitting, resulting in unstable predictions and too optimistic model performance measures. The reported AUC of 0.92 is therefore very likely to be an overestimation.

    Second, the SPOT algorithm provides a treatment advice based on the predicted outcome after treatment, without providing the...

    Show More
    Conflict of Interest:
    Personal disclosures
    FK None reported.
    E. Venema, H.F. Lingsma, and D.W.J. Dippel were involved in the development of a prediction model (MR PREDICTS).

    Venema E, Mulder MJHL, Roozenbeek B, Broderick JP, Yeatts SD, Khatri P, et al. Selection of patients for intra-arterial treatment for acute ischaemic stroke: development and validation of a clinical decision tool in two randomised trials. BMJ. 2017;357.