The Journal of Korean Medical Science (JKMS) is a globally recognized periodical published by the Korean Academy of Medical Sciences. Founded in 1986, it is now publishing its 37th volume.1 The JKMS is a flagship journal in Korea covering all subjects in general medicine and serving a model of quality editing and publishing for other scholarly journals in the country and related region.2 The Editor-in-Chief of JKMS has been a member of the International Committee of Medical Journal Editors (ICMJE) since 2016.

JKMS is a platform for publishing quality research reports and other scientific documents that contribute to the evidence-based data and scientific knowledge dissemination. It employs various open-access channels, including social-media platforms, to reach the global audience, maintain post publication communication, and attract scientific credits.3 The most reliable scientific credit for scholarly articles and journals is perhaps citations which are often viewed as hard currency for the publishing enterprise.

While journal editors are responsible for most, if not all, aspects of their published contents, they should also employ all ethical strategies for attracting relevant citations and advantageously ranking their journals. One of the successful strategies is inviting great articles with potentially societal and citation impacts.

The tough publishing competition has led to selective and at times coercive citation practices across some journals which are widely known for their ‘citation cartels.’ Notably, citation activities in the ‘Big Six’ (B6) journals in general medicine were analyzed in a remarkable study that reported on low self-citations, with an exception of the New England Journal of Medicine where clinical trial reports had abundance of journal self-citations.4 The share of self-citations in the B6 trial reports ranged from 14.9–23.1% with an average of 18.4% and the same in reviews ranged from 10.5–27.6% with an average of 11.9%. Such auto-citations in the B6 journals may be due to a selective approach to quality and prestige of high-impact sources, tempting all stakeholders in the publishing to cite the most impactful items.4

Citation analyses may visualize loopholes in the publishing enterprise and prompt the journal editors to upgrade their policies. Over the past decade, JKMS has been actively implementing the best standards in reviewing, editing, and publishing to contribute to the science growth globally. And it is essential to overview the accumulated citation data and highlight areas where citations are missing.5

There are several global databases and platforms that track and record citations: Web of Science (WoS, Clarivate^®), Crossref, Scopus, and PubMed Central (PMC). We chose WoS as the most selective and prestigious platform to analyze citation activity and related trends for JKMS articles published in 2011–2020.

The JKMS publications in 2011–2020, excluding editorial and correspondence items, were analyzed in view of their citations on WoS from December 2021 to June 2022. Citations by year, individual article, article type, and scientific category were analyzed to demonstrate citation trends. Citations of same articles were also retrieved from Crossref, Scopus, and PMC on the same day to reveal any differences between them.

Statistical significance was analyzed using χ² test or Fisher’s exact test.

A total of 2,880 articles were published in JKMS from 2011 to 2020 and 2,757 (96%) of them were cited more than once (Table 1). A few research articles received no citations during 5 years post publication. All review articles have been cited. Brief communications (89/97) and images (20/31) were less cited than others (Table 1).

There are no criteria of waiting time duration to declare non-citation, but the cited half-life of 5.4 years by Journal Citation Report (JCR^®) 2021 may suggest that articles in JKMS have much less possibility of citation after 5 years from publication. Based on the 5-year half-life, non-citation data of JKMS articles in 2010–2015 can be summarized in Table 2. The overall non-citation rate was 1.4% and that for research articles was 1%.

Articles in endocrinology, gastroenterology, human genetics, immunology, laboratory medicine, medical informatics, medical imaging, musculoskeletal disorders, and urology have been cited 97% or more. In contrast, articles in biomedical engineering showed 75% citation rate, the lowest by category (P = 0.020). Articles in Obstetrics and Gynecology, Otorhinolaryngology, and Surgery were 89–90% less cited than those in other categories but not significant statistically (Table 3).

The numbers of cumulative citations were depicted chronologically by dates of publication in Fig. 1. Articles cited more than 100 times were published from 2012 onward, with citation numbers steadily increasing from the same year. Importantly, 10 articles attracted more than 100 citations in 2020.

Citation counts were analyzed in connection to period post publication (Fig. 2). Articles in 2011–2014 showed similar level and pattern of citation during 2–5 years whereas those in 2015–2018 rapidly received more citations during 1–5 years post publication. Citations to articles in 2020, mostly related to severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and coronavirus disease 2019 (COVID-19) skyrocketed in 2020 and 2021 (Figs. 1 and 2).

Total citations to 2,880 JKMS articles in 2011–2020 were 33,254, 31,512, and 33,129 by WoS, Crossref, and Scopus, respectively (Table 4). The same record by PMC was 18,984. Annual citation activity recorded by all databases demonstrated similar patterns for a period between 2011 and 2020. The decreasing average count per article in 2011 was 17.54 by WoS, 16.31 by Crossref, and 17.48 by Scopus, and the counts slowly decreased later year by year until 2019 (Table 4).

The current citation analysis demonstrated that the overall citation rate of JKMS articles published in 2011–2020 was 96% (as of December 2021; Table 1). The aggregation of citations to JKMS articles requires some time post publication, with an initial citation growth and a decline over time. Due to the current rapid development trends in clinical research, the initial citation activity is more informative and suggestive of ‘hot’ topics than the delayed activity. This is why cited half-life by JCR^® may help reveal trendy topics and their attractiveness over time. As a prime example, the cited half-life of JKMS articles was reported 5.4 years by JCR^® 2021. Supposedly, JKMS articles are less likely to attract new citations after 5 years post publication. If the cited half-life is known for a journal, it may be used as time lag duration to estimate non-citation rate after publication.

Only one to four research articles published annually in 2010–2016 did not receive citations. The same figure increased from 2017 onward, with 11 articles (7 research articles) remaining without citations by 2021 (Tables 1 and 2). The non-citation rate is one of the proxy parameters of journal quality which is negatively correlated with the Journal Impact Factor (JIF) by JCR^®.5 Skewed citations and non-citation are well-known patterns of all journals, but high-impact sources often escape non-citation.56 Given the cited half-life 5.4 years recorded by JCR^®, the time lag for estimation of non-citation rate of JKMS can be 5 years post publication. The non-citation rate of all 2011–2016 JKMS articles stands at 1.4% and the same for research articles at 1%.

By article type, all review articles, including those in 2020, received citations, but only 96% of research articles were cited. Interestingly, 10, 5, and 45 research articles published in 2018, 2019, and 2020, respectively, received no citations by December 2021. With the cited half-life of 5.4 years, non-cited articles may still receive citations in the foreseeable future.

While brief communications and opinion pieces in JKMS attracted low citations, image articles (images in this issue) showed the lowest citation rate of 65%. Clinical trial reports were cited as actively as other research articles in JKMS, something in contrast to the B6 journals where trials had blockbuster effect.4 Importantly, JKMS reviews and special articles demonstrated high citability, pointing to the necessity to actively solicit, edit, and publish these articles to boost JKMS citation rates.

JKMS articles cover 36 scientific specialties. Most of the article categories demonstrated 90% and more citation rates. Only biomedical engineering articles resulted in the significantly least cited proportion (75%). The biomedical engineering section was introduced at JKMS in 2017, and this section may require more time to mature and attract healthy flow of citations. Possibly, there is still not enough readers and citers for this section, pointing to the need for more active post publication promotion.

Articles in obstetrics and gynecology, otorhinolaryngology, and surgery demonstrated 89–90% citation rates but without statistical significance. Articles with too specific (narrow) topics were less likely to be cited which might be due to the small pool of interested researchers. Most submissions with too narrow scope are usually rejected outright at JKMS; and only handful of these submissions pass peer review and get published. Examples of non-cited articles in JKMS include those on health-related laws or compensation strategy in connection to industrial hazards, and some clinical studies with narrow topics. Apparently, the current citation analysis suggests that JKMS editors should prioritize submission, publication, and post publication promotion of items within this journal’s general medical scope.

Total citations to JKMS articles in 2011–2014 were around 300 one year post publication and 500 after two years post publication. Compared to this, articles in 2015–2019 received 400–500 and 600–800 citations, respectively (Fig. 2). This trend can be viewed as a boost in citation activity. As a whole, total citations increased slowly and steadily from 2011 to 2019. The transition of the journal from monthly to weekly publication schedule in 2018 had a negligible immediate influence citation-wise.

Understandably, articles in 2020 were quite different from previous sets. The articles published in 2020 were cited 1,421 times in 2020, and 2,470 times in 2021. In 2020, JKMS published 378 influential articles, including 126 on SARS-CoV-2 or COVID-19 pandemic. The pandemic articles had a booster effect on citation counts. Thanks to these, JKMS recorded its highest JIF 2021 by JCR^® at 5.354 (2.153 in 2020). The big jump was partly due to the weekly publication schedule since 2018 which allowed rapidly publishing numerous Korean articles on SARS-CoV-2 or COVID-19 and timely exposing them to the global scientific community. The current achievement needs to be nurtured to raise JKMS standards and scope further to the level of a global general medical journal.

Table 4 data demonstrated similar level of citations to JKMS articles recorded by WoS, Crossref, and Scopus. We can rely on any convenient data from these platforms to estimate citations of JKMS articles. Only the numbers from PMC were twice less than those from either WoS, Crossref, or Scopus. Such a difference is possibly due to the small total number of journals archived by PMC (i.e., 2,636 full participating journals and 307 National Institute of Health portfolio journals).7 Although PMC covers 8,668 selective deposit journals, these are only for selective open-access articles. Compared to this, WoS and Scopus cover substantively more active sources. One of the previous reports recorded less citations for JAMA, Lancet, and New England Journal of Medicine in WoS than in Scopus and Google Scholar.8 The difference could be due to the selectiveness of WoS. Google Scholar tracks all scientific information in the Internet without any selectiveness and thus its recorded citation numbers could be much higher than those by citation-tracking bibliographic databases. As such, Google Scholar counts were not analyzed in the current study.

The current analysis did not reflect on social media metrics. JKMS launched its Twitter channel in May 2019 and started actively promoting selected articles. It is hoped that dissemination of information via the official channels may enable attracting more interested readers and those who consider our precious articles for citations. Dissemination of research findings via social media has been found beneficial for raising citations across some disciplines.39 Related social-media analyses are warranted for further updating JKMS editorial policies.

In conclusion, articles in JKMS have accrued citations similarly between scientific categories except for those in biomedical engineering. Articles have been cited continuously during 5 years post publication, and those from 2016 received citations more than those before 2015. Review articles are usually well cited. Non-citable articles are often those with narrow scope and specific focus. Articles on SARS-CoV-2 or COVID-19 in 2020 enabled JKMS to boost its citation metrics, particularly the JIF 2021. It is hoped that the current citation analysis will guide the journal editors for adjusting their daily practices and upgrading the publishing standards.