PDF
ePub
Citation
Print
INTRODUCTION
The purpose of scientific journals is to share new information and novel discoveries with professionals of the relevant specialty. This is critical in medicine, where the application of new knowledge from journals advances clinical practice. For this reason, the timeliness of research data is essential; indeed, some journals provide submitting authors with specific guidelines in this regard (1). Moreover, in radiology, data timeliness may be especially important since the field relies heavily on new technology particularly digital technology (234567), which develops and changes faster than in other disciplines. Thus, the timeliness of data in radiology journals may be an important indicator of quality or impact, and the present study analyzed and compared several representative general radiology journals in terms of the age of the data published therein.
MATERIALS AND METHODS
This study did not require an Institutional Review Board approval.
Study Selection
We analyzed the Korean Journal of Radiology (KJR), European Radiology (ER), and Radiology, which have the highest impact factors among general radiology journals published in Asia, Europe, and America, respectively, according to the 2017 Journal Citation Reports (Clarivate Analytics, Philadelphia, PA, USA). We hand-searched the print issues of the journals published in 2017 (6 issues of KJR, 12 issues of ER, and 12 issues of Radiology) to find reports of original research studies that had collected and analyzed patient data. Experimental studies involving animals, phantoms, or other laboratory conditions were excluded. The literature was screened by two board-certified radiologists, both of whom were proficient in reviewing radiology research articles.
Data Extraction
Firstly, the eligible articles were reviewed to determine whether they had stated the start and end of the data collection period to the level of the calendar month (Fig. 1). The data collection period was defined as the time from recruitment of the first patient to the latest follow-up of the study patients, as defined elsewhere (Fig. 1) (8). We considered month, but not date, as it is rare for published radiology research studies to report the data collection period to the level of date. Next, we made a universal assumption that the start and end dates of the data collection period fell on the first and last days, respectively, of the reported months. For example, if the data collection period ranged from January 2011 to April 2015, the start and end dates were assumed to be January 01, 2011 and April 30, 2015, respectively. As most studies only report the shortest and longest follow-up durations, rather than the exact follow-up periods of each study participant, the exact time of each patient's latest follow-up is rarely given. Therefore, we determined the time of the latest follow-up by 1) assuming that the earliest study enrollee had the longest follow-up and that the last enrollee had the shortest follow-up, 2) calculating two time points by adding the longest and shortest follow-up lengths to the start and end, respectively, of enrollment, and 3) choosing the later of these two time points. This method gave a reasonable estimate of the latest follow-up time. Secondly, the type of study design was determined (retrospective, prospective, or unclear). Thirdly, the exact date of publication, according to the time at which the full-text article became available online, was recorded by referring to the PubMed data of ER and Radiology, both of which use electronic publication ahead of print, and through contact with the editorial office of KJR, which does not use electronic publication ahead of print. Two board-certified radiologists analyzed the eligible articles to extract data. When there was ambiguity, a third reviewer experienced in the relevant methodology was invited.
Outcome Measures and Statistical Analysis
The proportion of articles that revealed the data collection period to the level of the calendar month was calculated, along with its 95% confidence interval (CI). The proportion was compared between journals in a pairwise manner using Fisher's exact test: KJR vs. ER, KJR vs. Radiology, and ER vs. Radiology. In articles that revealed the data collection period, the age of the research data was calculated as the time between the end of the data collection period and the date of publication (Fig. 1). The distribution of the age of data was checked using a histogram and the Kolmogorov-Smirnov test, and appropriate summary statistics were obtained. Next, the age of the data was compared between journals in the aforementioned pairwise manner using the Wilcoxon rank-sum test. This statistical comparison was performed across all relevant articles, as well as separately for prospective and retrospective studies. The threshold p value for statistical significance was lowered to 0.017 (Bonferroni adjustment) to maintain the overall alpha at 5% after the three pairwise comparisons were made. Statistical analyses were performed using SPSS Statistics for Windows, version 21.0 (IBM Corp., Armonk, NY, USA).
RESULTS
In total, 50 KJR (9101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354565758), 492 ER, and 254 Radiology articles reported original research studies analyzing patient data. Of these, 44 (88%; 95% CI: 75.8–94.8%) (910121314151617181920212324262728293031323435363738394041424344474849505152535455565758), 359 (73%; 95% CI: 68.9–76.7%), and 211 (83.1%; 95% CI: 78–87.2%) articles, respectively, revealed the start and end of data collection to the level of calendar month. The point estimate value of this proportion was slightly larger in KJR than in ER and Radiology, although the difference was not statistically significant. In contrast, ER demonstrated a significantly lower proportion than Radiology in this regard (Table 1). Further breakdowns according to study type are also presented in Table 1, and the separate results from prospective and retrospective studies were mostly consistent with the overall results.
The age of data (the time between the end of the data collection period and the date of publication) was skewed to the left in all three journals (Fig. 2) and was significantly larger in KJR (median age: 826 days across all relevant articles) than in ER (median age, 570 days) and Radiology (median age: 618 days) (Table 2). The difference was more pronounced when retrospective studies were considered separately (Table 2).
DISCUSSION
Korean Journal of Radiology seemed not to fall behind ER and Radiology regarding the proportion of articles that reported the data collection period, although the results of prospective studies may be inconclusive because too few relevant articles were published in KJR (11 articles). Nonetheless, this result may indicate that the journal has good quality control in the peer review and editorial processes. However, the age of data was significantly greater in KJR than in ER (approximately 8.5-month difference in median age) and Radiology (approximately 7-month difference in median age). Furthermore, the greater age of the data was more pronounced when retrospective studies were considered separately (approximately 13 months older than in ER and 11 months older than in Radiology). It is likely that the KJR contains older data because authors generally submit their manuscripts to higher-ranked journals first, descending the ranks of journals if their manuscript is rejected. As a result, lower-ranked journals would naturally contain older data. The journal impact factors of KJR, ER, and Radiology for 2017, according to the Journal Citation Reports (Clarivate Analytics), were 3.072, 4.027, and 7.469, respectively. This would explain why KJR differs from ER or Radiology in this regard.
However, there was no significant difference in the age of data between ER and Radiology, despite the apparent difference in journal impact factor, perhaps because ER publishes accepted articles more swiftly than Radiology in electronic publication format ahead of print publication. Specifically, in the articles analyzed in the present study, the median interval between initial electronic publication and official assignment to a monthly print issue was 225 days (range: 130–316 days) for ER and 138 days (range: 39–244 days) for Radiology.
Korean Journal of Radiology recently assessed the quality of its research articles in terms of conformity to the Standards for Reporting of Diagnostic Accuracy Studies (STARD) 2015 guidelines (59) and the adequacy reporting reliability analysis for diagnostic tests (60). The current study revealed another area in which KJR could improve its quality and impact, and we would even suggest some specific measures. Firstly, because the age of data in the KJR differed more markedly from that in ER and Radiology in retrospective studies than in prospective studies, the journal could encourage authors to make data as recent as possible by updating study data in the revision process. Such an update would be possible in the case of retrospective studies, although it is likely infeasible in most prospective studies. Secondly, the journal could further shorten the time from initial submission to publication by ensuring more rapid review, allowing electronic publication before print, and publishing monthly in a smaller volume. Indeed, bimonthly publication delays those articles that were early accepted in the 2-month cycle period.
This study had several limitations. Firstly, as studies generally do not report each individual's exact follow-up duration, we assumed that the earliest study enrollee had the longest follow-up and the latest enrollee had the shortest follow-up; this may not always have been the case. However, we believe that this approach gave a reasonable estimate of the data collection period and was sufficient to analyze the macroscopic, between-journal difference. Secondly, one published study (8) defined the age of data slightly differently from the present study as the time from the mid-point of the data collection period to the publication date. This previous study analyzed prospective randomized trials, wherein their definition fit better. However, this definition would have been inappropriate in our study, wherein the majority of studies analyzed were retrospective. In retrospective studies, a data collection time extending further into the past may be more beneficial. However, for the purposes of the present study, it may have penalized the results, as the data age became greater. Thirdly, knowledge of the details of any rejections by other journals (number of rejections, by what journals, etc.) before submission to KJR would have been helpful in further understanding the greater age of KJR data. However, we empirically found that it was difficult to collect such “sensitive” information with consistency.
In conclusion, KJR did not fall behind ER or Radiology with regard to the proportion of articles that reported the age of data. However, the age of data was significantly greater in KJR than in ER and Radiology, suggesting that the journal should introduce some measures to improve the timeliness of data that it publishes.
XML Download