Background and purpose Alternative metrics (altmetrics), based on mentions in online media, is a new tool that can help to identify the most influential articles to diverse public audiences. This article aimed to determine the 100 most mentioned articles in the field of neurointervention and to analyze their characteristics.
Materials and methods We selected the 808 journals that were considered journals potentially publishing articles on neurointervention. We also selected articles using keywords. Using the Altmetric.com search tool, we identified the 101 most mentioned neurointervention articles based on the highest altmetric attention scores (AASs) within selected journals and articles. Each article was evaluated for several characteristics including AAS, number of citations, journal title, journal category, impact factor of the journal, year of publication, authorship, country, type of document, and topic.
Results The AASs for the top 101 articles ranged from 1586 to 39. Stroke published the largest number of articles (19.8%) followed by the New England Journal of Medicine (17.8%). The majority of articles were published in multidisciplinary journals (38.6%), were published in 2017–2018 (43.6%), originated from the USA (54.5%), were original articles (66.3%), and dealt with intra-arterial thrombolysis or thrombectomy for acute ischemic stroke (58.4%). Tudor G Jovin was the most prolific author, authoring 18 of the most mentioned neurointervention articles
Conclusions This study presents a detailed list of the 101 most mentioned neurointervention articles in online media, thus providing useful information on the dissemination of neurointervention research to the general public.
Statistics from Altmetric.com
Accurate measurement of the impact of scientific articles is important for guiding decisions in promotions, tenure, and funding, and for identifying the most influential research within a particular field. Various bibliometric indices are now used to analyze the quantity and quality of scholarly research.1 The number of citations is the most widely used indicator of the impact of individual research articles.2 However, traditional citation analysis has been heavily criticized because citation requires a significant duration of time after publication and reflects the impact of an article among essentially researchers in the same field.3
The recent advent of the internet and the growth of social media have greatly promoted the rapid dissemination of scientific knowledge among more diverse audiences.3 The concept of ‘altmetrics’ (referring to non-traditional alternative article-level metrics) was implemented in 2010 in reference to new metrics that evaluate the early influence of scholarly material on broad audiences in comparison with citation counts.4 Altmetrics measure scientific and social impacts of an article after publication, based on the number of ‘mentions’ of an article across various online sources.4 5
Neurointervention is a relatively new medial subspecialty that uses minimally invasive techniques, principally to treat disorders of the blood vessels of the brain, head and neck, and spinal cord.6 There have been several studies to identify the most mentioned articles from online media in various medical disciplines, including dentistry,7 emergency medicine,8 and neurosurgery.9 To our knowledge, however, the bibliometric analysis of the most mentioned articles in the field of neurointervention has not yet been reported.
The purpose of the present study was therefore to identify the 100 most mentioned neurointervention articles and to analyze the characteristics of these articles.
Materials and methods
The present study did not involve human subjects and thus did not require approval from an institutional review board.
Selection of journals and articles
To identify the most mentioned neurointervention articles, journals listed under the following six subject categories of the Web of Science (Thomson Reuters, New York, NY) for the year 2016 were evaluated for inclusion: ‘clinical neurology’ (194 journals), ‘medicine, general and internal’ (155 journals), ‘neuroimaging’ (14 journals), ‘neuroscience’ (259 journals), ‘radiology, nuclear medicine and medical imaging’ (127 journals), and ‘surgery’ (197 journals). Subject category of ‘medicine, general and internal’ was included in our study because it contains multidisciplinary journals that cover a broad spectrum of scientific research. An additional search was performed on the Web of Science for journals including the title word ‘neuro*’ or ‘intervention*’ (an asterisk replaces any number of characters) regardless of the subject category, and 296 journals were identified. After excluding 434 duplicate journals, a total of 808 journals were considered journals potentially publishing neurointervention articles and were included in our study.
In addition, we also searched for neurointervention articles using the following keywords: angioplasty, aneurysm, AVM, AVF, balloon, CCF, circle of Willis, coil, embolization, endovascular, flow diverter, guide wire, microcatheter, neurointervention, Onyx, perfusion, recanalization, retriever, Solitaire, stent, stroke, thrombectomy, and thrombolysis.
Identification of the 100 most mentioned neurointervention articles in online media
Each of the journals and articles was searched separately for the altmetric attention score (ASS) of each article using the ‘advanced search’ in the Altmetric Explorer (https://www.altmetric.com/explorer, Altmetric LLP, London, UK). The site was accessed on a specific day, May 25, 2018, to avoid changes in online activity of articles.
Altmetric.com, one of the main providers of alternative indicators, was chosen for the study because it is the most comprehensive source covering the vast majority of online media activity associated with scientific papers. Altmetric.com has captured real time online mentions of published articles from public policy documents, blogs, mainstream media, Wikipedia, online reference managers such as Mendeley, research highlights, postpublication peer review platforms, Open Syllabus, YouTube, and social media networks, including Facebook and Twitter, since 2011.4 Altmetric.com provides the AAS to measure the overall level of online impact arising from a particular research output, which is presented as a whole number. The AAS is a weighted score of total mentions of the article across various online media, reflecting the anticipated relative degrees of influence of sources on potential readers (eg, default weights of 8 for news outlet, 5 for science blog, 3 for Wikipedia or policy document, 1 for Twitter, and 0.25 for Facebook).4
All articles were recorded and compiled into a single database. These articles were then ranked in descending order based on the ASS and reviewed to determine if they were relevant to neurointervention. A neurointervention article was defined as any study which met all of the following three conditions: (1) focused on techniques or outcomes of endovascular treatment for cerebrovascular disease; (2) described interventional procedures of the intracranial arteries, extracranial portion of the internal carotid artery and vertebral artery, external carotid artery and its branches, common carotid artery, spinal artery, or intracranial veins; and (3) had any impact or relevance on neurointerventionalists' daily clinical practice. Articles were excluded if they explored basic science research (ie, hemodynamics, pathophysiology, or biochemistry), dealt purely with surgery or conservative treatment, or focused on diagnostic imaging to evaluate an indication or outcome of neurointerventional treatment.10 No restrictions were placed on the document type, language, or scholarly identifiers for our study.
Analysis of articles
We analyzed the full text of the most mentioned articles and extracted the following information: (1) ASS, (2) number of citations (obtained from the Web of Science database, Thomson Reuters, New York, NY, USA, retrieved on July 10, 2018), (3) journal title; (4) journal category (basic science, neurology, neurosurgery, radiology, cardiology, surgery, otorhinolaryngology, or multidisciplinary); (5) impact factor of the journal (based on the 2017 science edition of the Web of Science); (6) year of publication; (7) authorship; (8) country; (9) type of document (original article (reports that investigated clearly stated objectives or hypotheses and contained specifically articulated methods and results sections), review, technical note, guideline/consensus statement, systematic review/meta-analysis, case report, news, or letter); and (10) topic.
For the purpose of our study, the country of origin was defined by the affiliation provided for the first author. If the first author had affiliation to more than one country or group name, the corresponding author’s affiliation was used for the origin of the article.
Two reviewers (JYL and HJJ, neurointerventionalists with 15 and 10 years' experience, respectively) independently conducted identification and analysis of the articles. In the case of disagreement between the two reviewers, a consensus was achieved by open discussion.
The present study adopted a descriptive research approach by means of bibliometric analysis.
The online supplementary table lists the 101 most mentioned neurointervention articles (the three 99th articles had an equal AAS) in descending order, according to their AASs, obtained from the Altmetric.com database at the time of the analysis. The altmetric scores of the 101 most mentioned articles ranged from 1586 to 39 (mean 191.4±270.8; median 82). The number of citations of these articles ranged from 1744 to 0 (mean 139.7±346.1; median 9). The most mentioned neurointervention article was the 2018 paper by Nogueira et al, ‘Thrombectomy for stroke at 6 to 24 hours with selection by perfusion imaging’, in the New England Journal of Medicine.11 The second most mentioned article was ‘Thrombectomy for stroke at 6 to 16 hours with selection by perfusion imaging’, also published in the New England Journal of Medicine in 2018 by Albers et al.12 These two papers described the effect of endovascular thrombectomy performed more than 6 hours after the onset of ischemic stroke.
Supplementary file 1
The most mentioned articles were published in 25 journals, led by Stroke (n=20), followed by the New England Journal of Medicine (n=18). The leading journal categories were multidisciplinary (38.6%), followed by neurology (37.6%) and neurosurgery (10.9%) (table 1).
All articles were published between 2010 and 2018, with 43.6% of articles being published in 2017–2018 (table 2). Table 3 presented a list of authors, which was led by Tudor G Jovin, who authored 18 of the most mentioned neurointervention articles, and Jeffrey L Saver, who authored 15 articles. The 101 articles originated from 16 different countries: the majority of the articles originated from the USA (54.5%), distantly followed by Canada (9.9%), and The Netherlands (6.9%) (table 4).
In terms of the type of document, original articles comprised 66.3% of the most mentioned articles, of which 38.6% were comparative clinical trials and 26.7% were clinical observational studies (table 5). The most common topic was intra-arterial thrombolysis or thrombectomy for acute ischemic stroke (58.4%), followed by angioplasty or stent for extracranial carotid artery stenosis (13.8%) (table 6).
With the advent of the internet and social media, interest in altmetrics has grown rapidly since the concept was first introduced in 2011.13 Altmetric.com is one of the first companies to work in this area, which contains more than 21 million research outputs and captures real time mentions in online media. Altmetric.com automatically calculates an altmetric indicator, the AAS, which is a weighted count of all mentions a scholarly article has received across various online sources.4
The AAS for the 101 most mentioned articles ranged from 1586 to 39, which was higher than that for neurosurgery (AAS 643–45) reported by Wang et al.9 The higher score of neurointervention articles in our list might reflect the fact that we searched for articles from 808 journals of various disciplines whereas Wang’s study limited the search to 18 neurosurgical journals. In addition, we also used search terms related to neurointervention and therefore may have included the most influential neurointervention articles that were published elsewhere. Indeed, citation rates differ for each specialty and might depend on the size of the scientific community. In contrast, altmetrics may depend on not only the size of the research fields but also interest from the general population.
The number of citations of the most mentioned articles were relatively low, ranging from 1744 to 0 (mean 139.7) and 18 articles had an AAS of zero. A previous citation classic study published in 2017 reported the 100 most cited neurointervention articles that were cited between 1912 and 170 times (mean 363.4).10 Only five articles overlapped between the 101 most mentioned articles in our study and the 100 most cited articles in the previous work.
There have been several studies examining the relationship between citation rates and altmetric indicators for scientific papers. Most studies have shown very weak correlations between citation rates and altmetric measures.14–16 Recently, two studies correlated the citation rates and AASs for the most cited articles in specific medical fields and also found weak positive correlations.8 17 These low correlations suggest that altmetrics operate independently of citations, and thus can be used as an alternative measure of research impact. It is reasonable to use the number of citations to assess the ‘academic impact’ of a scholarly article. Altmetrics, on the other hand, appear to reflect the ‘disseminative impact’ based on public interest, rather than the scientific merits of an article.4 It is worth noting that several non-scientific factors may influence the AAS of scientific articles. For example, articles with simple and declarative titles were more likely to have high altmetric scores irrespective of their scientific impact or importance.18
Recently, Kim et al performed a bibliometric analysis of the 100 most cited articles in the field of neurointervention.10 Although a direct comparison is limited because of differences in time of data collection, we observed several different trends using alternative metrics to those identified in a previous citation study.
First, our analysis indicated that 38.6% of the most mentioned neurointervention articles were published in multidisciplinary medical journals (ie, the New England Journal of Medicine, The Lancet, and the Journal of the American Medical Association). According to the previous study, 18.0% of the most cited articles were published in multidisciplinary journals.10 These prestigious journals have higher impact factors and a much wider audience than specialty journals. Although neurointervention has rapidly developed over the past decades, it is still a small field compared with others, and is unfamiliar to the public. Therefore, it is unsurprising that an article appearing in a high impact multidisciplinary medical journal receives more interest from the public when compared with an article published in a specialty journal.
A large proportion of medical journals are utilizing social media platforms such as Twitter and podcasts as a means of rapid dissemination of knowledge within and outside the academic community.19–21 In addition, it is evident that the use of a dedicated Twitter profile was shown to have a great effect on a journal’s impact factor.22 The journal editors and publishers may have to consider the usage of non-traditional methods to expand the journal readership and increase the impact of research articles.
Second, our study found that the most mentioned neurointervention articles were recent; only 15.8% of articles were published before 2015 and 43.6% of articles were published in 2017–2018. This finding emphasizes that altmetrics is particularly sensitive to recent news, and more recent publications receive higher AASs. In contrast, the previous citation study reported that 88.0% of the most cited articles were published before 2010.10
Third, our study showed that 54.5% of the most mentioned articles originated from the USA, which is in agreement with but slightly more concentrated in the USA compared with the previous citation classic study in the same field (45.0% US articles).10 These findings prove not only scientific impact but influence beyond academia of the USA in research related to neurointervention.
Fourth, regarding the document type of articles, original articles accounted for 66.3% of the most mentioned neurointervention articles. This result is similar but not identical to findings from the previous citation study that found 95.0% of the most cited neurointerventional articles were original articles.10 This result confirms that document type influences the alternative metrics of publications. Although original articles attract the largest audiences, articles with other types may also receive attention from widely diverse audiences that use social media to follow the biomedical literature, whereas citations are concentrated mainly on original articles.
Finally, the most popular topic in our study was intra-arterial thrombolysis or thrombectomy, which comprised 58.4% of the most mentioned articles. In contrast, only eight articles were related to endovascular treatment of cerebral aneurysms, which was the most common topic (42.0%) discussed in the most cited articles.10 The increasing public awareness of neurointerventional treatment for acute ischemic stroke, propelled by dissemination of knowledge through popular and social media, may be the reason for increased public engagement in the research for intra-arterial thrombolysis or thrombectomy. In addition, significant technical advancements of intra-arterial thrombolysis or thrombectomy and their new applications for acute ischemic stroke may also increase the influence and reach of articles on this topic.
The present study has some potential limitations. First, there are inherent shortcomings that should be considered in the use of altmetrics. Altmetrics do not cover the demographics of those mentioning online research material (professional or public) and the nature of each mention (positive or negative). Furthermore, the credibility of commentators and the validity of their comments in online media is hindered due to the anonymity of the online world.23–25 Second, only data supplied by Altmetric.com was used for assessing alternative metrics. Other tools, such as Plum Analytics, ImpactStory, and ALM-PLoS, also provide article-level alternative metrics that use different online sources and algorithms, which may lead to different results.26
This study presents a detailed list of the 101 most mentioned neurointervention articles in online media by using the Altmetric.com database. Our findings provide useful information on the dissemination of neurointervention research within the public domain.
Contributors HJK: study design, data acquisition, data analysis, data interpretation, and manuscript preparation. DYY: study design, data acquisition, data analysis, data interpretation, and manuscript revision. ESK: data acquisition and data analysis. EJY: data interpretation and manuscript revision. HJJ: data acquisition, data analysis, and data interpretation. JYL: data acquisition, data analysis, and data interpretation. B-MC: data analysis and data interpretation.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial, or not-for-profit sectors.
Competing interests None declared.
Patient consent None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Correction notice Since this paper was first published the results section has been amended. The reference to the paper title ’Thrombectomy for stroke at 6-16 hours…' has been changed to ’Thrombectomy for stroke at 6-24 hours…'
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.