Journal List > Korean J Radiol > v.17(5) > 1027262

TOOLS
Choi, Chung, Koo, Park, Yoon, and Park: Does the Reporting Quality of Diagnostic Test Accuracy Studies, as Defined by STARD 2015, Affect Citation?
Experimental and Others
Original Article

Korean Journal of Radiology 2016; 17(5): 706-714.

Published online: 23 August 2016

DOI: https://doi.org/10.3348/kjr.2016.17.5.706

Does the Reporting Quality of Diagnostic Test Accuracy Studies, as Defined by STARD 2015, Affect Citation?

Young Jun Choi, MD, Mi Sun Chung, MD, PhD, Hyun Jung Koo, MD, Ji Eun Park, MD, Hee Mang Yoon, MD, Seong Ho Park, MD, PhD

Department of Radiology and Research Institute of Radiology, University of Ulsan College of Medicine, Asan Medical Center, Seoul 05505, Korea.

Corresponding author: Seong Ho Park, MD, PhD, Department of Radiology and Research Institute of Radiology, University of Ulsan College of Medicine, Asan Medical Center, 88 Olympic-ro 43-gil, Songpa-gu, Seoul 05505, Korea. Tel: (822) 3010-5984, Fax: (822) 476-4719, parksh.radiology@gmail.com

Received 28 May 2016       Accepted 29 May 2016

Copyright © 2016 The Korean Society of Radiology

(open-access):

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Objective

To determine the rate with which diagnostic test accuracy studies that are published in a general radiology journal adhere to the Standards for Reporting of Diagnostic Accuracy Studies (STARD) 2015, and to explore the relationship between adherence rate and citation rate while avoiding confounding by journal factors.

Materials and Methods

All eligible diagnostic test accuracy studies that were published in the Korean Journal of Radiology in 2011–2015 were identified. Five reviewers assessed each article for yes/no compliance with 27 of the 30 STARD 2015 checklist items (items 28, 29, and 30 were excluded). The total STARD score (number of fulfilled STARD items) was calculated. The score of the 15 STARD items that related directly to the Quality Assessment of Diagnostic Accuracy Studies (QUADAS)-2 was also calculated. The number of times each article was cited (as indicated by the Web of Science) after publication until March 2016 and the article exposure time (time in months between publication and March 2016) were extracted.

Results

Sixty-three articles were analyzed. The mean (range) total and QUADAS-2-related STARD scores were 20.0 (14.5–25) and 11.4 (7–15), respectively. The mean citation number was 4 (0–21). Citation number did not associate significantly with either STARD score after accounting for exposure time (total score: correlation coefficient = 0.154, p = 0.232; QUADAS-2-related score: correlation coefficient = 0.143, p = 0.266).

Conclusion

The degree of adherence to STARD 2015 was moderate for this journal, indicating that there is room for improvement. When adjusted for exposure time, the degree of adherence did not affect the citation rate.

Keywords: STARD, STARD 2015, Citation, Reporting quality, Accuracy, Adherence, Impact, Impact factor

INTRODUCTION

The quality of scientific research articles that are published in a journal consists of two elements, namely, the quality of the report and the quality of the science. These quality variables do not necessarily concur, although they do overlap. In other words, a well-reported study does not necessarily mean that the study is of good scientific quality while a poorly reported study does not necessarily mean that the scientific quality of the study is also poor. Nevertheless, the quality of the report is very important because poor reporting quality hinders the ability of the readership to understand the authenticity, integrity, quality, and clinical impact of a research study. These deficiencies also hamper the effective generation of systematic reviews, which subsequently impacts the development of clinical guidelines and ultimately influences patient care (1). The great need for quality reporting in clinical research explains why there are longstanding and ongoing international efforts to create and promote standardized reporting guidelines for research studies, such as those reported by the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) network (1). The subject of the present study is the Standards for Reporting of Diagnostic Accuracy Studies (STARD) guideline, which was published in 2003 (2) and was recently updated to its second version, namely, STARD 2015 (34).
Peer-reviewed journals typically have limited space, and therefore publish far fewer articles than they receive. This continues to be true even in the present era of web-based electronic journals and open access journals (5). This space limitation means that journals try to select the highest quality articles for publication using editorial and peer reviews. However, it is well known that this review process can be deficient and subjective: these problems can result in the occasional publication of articles that are grossly deficient (678). One way to improve the objectivity and quality of the review process of journals may be to examine candidate articles for adherence to the appropriate standardized reporting guidelines. However, although this presumption may sound logical, there is as yet little robust evidence that supports it. Two studies have assessed whether the degree of adherence to STARD guidelines can be used as a metric of article quality. Specifically, they asked whether such adherence associated with the journal impact factor (9) or citation rate (10). However, the results were not conclusive; this partly reflected confounding due to impact factor differences between the journals that were included (10).
These observations led us to determine the rate with which original research studies of diagnostic test accuracy that were published in a single journal (the Korean Journal of Radiology [KJR]) complied with STARD 2015 guidelines (34). The fact that the articles were all from a single journal meant that our analysis of the relationship between adherence to STARD 2015 and citation rate was not confounded by journal factors.

MATERIALS AND METHODS

Literature Search and Study Selection

First, all original research papers that were published in KJR between January 2011 and December 2015 and whose abstract or title contained at least one of the following key terms were selected: "sensitivity", "specificity", "accuracy", "performance", "receiver operating", and "ROC". The full text of these articles was double-checked by one reviewer and one of four other reviewers to identify the articles that reported the diagnostic test accuracy of one or more tests relative to the reference standard in humans. All reviewers were experienced in diagnostic test accuracy studies and STARD as well as in radiological research in general.

Data Extraction

The five reviewers evaluated the eligible diagnostic test accuracy papers. Given that STARD 2015 had only been released just before this study started (34), a seminar attended by the five reviewers and an additional expert on literature review and bibliographic research was convened. The aim of the seminar was to review and discuss the items listed in STARD 2015, thereby ensuring that all of the reviewers had a clear understanding of STARD 2015.
Each of the five reviewers was randomly assigned a fifth of all eligible articles. They evaluated the articles independently according to the STARD 2015 checklist and extracted the relevant information, as explained below. Thereafter, any doubtful results were discussed at meetings attended by the five reviewers and the additional expert on literature review and bibliographic search until a complete agreement was achieved among all six people. Fulfilment of each STARD checklist item was recorded in a dichotomous manner, namely, yes (properly reported) or no (not reported). Of the 30 items in the STARD 2015 checklist (34), we excluded items 28 (registration number and name of registry), 29 (where the full study protocol can be accessed), and 30 (sources of funding and other support; role of funders) because items 28 and 29 had not been requested by the journal during the study period and item 30 only applies to funded research studies, which account for a minority of articles published in the journal. After completing this evaluation of the articles, the five reviewers also determined the total number of times each article had been cited by March 1, 2016, as indicated by the citation index reported in the Web of Science (Thomson Reuters, New York, NY, USA).

Statistical Analysis

The proportion (%) of articles that fulfilled each reporting item was determined. The total STARD score was then calculated for each article by adding the number of reported items. Thus, the maximum STARD score was 27 points (i.e., 30 minus 3 items). Assuming that each item is of equal importance, higher scores may roughly indicate better reporting quality. Regarding items 10, 12, 13, and 21, each of which consists of two sub-items (i.e., 10a, 10b, 12a, 12b, 13a, 13b, 21a, and 21b), each sub-item was given 0.5 points when it was fulfilled. This follows the practice established in previous similar studies (911). The mean and standard deviation (SD) of the total STARD scores were determined.
Of the 27 STARD items, 15 relate directly to the revised Quality Assessment of Diagnostic Accuracy Studies (QUADAS)-2 (12) tool for systematic reviews of diagnostic accuracy studies. QUADAS-2 consists of four key domains, namely, patient selection, the index test, the reference standard, and flow and timing. The 15 QUADAS-2-related STARD items are directly used to assess the risk of bias and study applicability and consist of STARD items 3, 6–13, 15, 16, and 19–22. Items 3, 6–9, 20, and 21 concern the patient selection domain of QUADAS-2, items 10a, 12a, and 13a concern the index test domain, items 10b, 11, 12b, and 13b concern the reference standard domain, items 19 and 22 concern the flow and timing domain, and items 15 and 16 concern both the index test and reference standard domains. The QUADAS-2-related yes/no score was calculated for each article, after which the mean (SD) QUADAS-2-related score of all articles was determined.
The articles were also divided according to whether they were cohort-type or case-control-type diagnostic accuracy studies. Cohort-type (single-gate) accuracy studies are characterized by the selection of subjects using one set of inclusion criteria, whereas, in case-control-type accuracy studies, the subjects are selected using multiple sets of inclusion criteria (131415). The mean (SD) total STARD, and QUADAS-2-related scores of these subgroups were calculated, and the two study types were compared in terms of these scores using the Student t test.
To assess whether the degree of fulfilment of the STARD 2015 items associated with the number of citations, a multivariable linear regression analysis was performed. The total STARD or QUADAS-2-related score served as an independent variable, exposure time (the time in months between publication and March 2016) served as a covariate to account for confounding by exposure time, and the number of citations served as the dependent variable.
All statistical analyses were performed using MedCalc for Windows (version 15.0; MedCalc Software, Ostend, Belgium), and p < 0.05 was considered statistically significant.

RESULTS

Subjects

The literature search strategy initially led to the identification of 123 articles. Of these, 60 were excluded because they were letters, editorials, or abstracts instead of a full article (n = 1), a case report or series (n = 2), a review article (n = 16), or not in the field of interest (n = 41). Thus, 63 articles fulfilled the eligibility criteria (161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778).

Adherence to STARD 2015 Items

The mean ± SD total STARD score (maximum of 27 points) of the included studies was 20.0 ± 2.1 (range 14.5–25). All 63 articles reported more than 50% of the items (total STARD score > 13.5), and 13 articles (20.6%) (16171819202122232425262728) reported more than 80% (total STARD score > 21.6). The mean ± SD QUADAS-2-related STARD score (maximum of 15 points) was 11.4 ± 1.7 (range 7–15). Sixty-one articles (96.8%) reported more than 50% of the items (QUADAS-2-related STARD score > 7.5), and 20 articles (31.7%) (1617181921222324252627345859616367717578) reported more than 80% (QUADAS-2-related STARD score > 12).
A closer assessment of each reporting item revealed that they varied widely in terms of rate of adherence (from 1.6% to 100%) (Table 1). Four items showed remarkably poor adherence rates (< 20%), namely, item 13b (whether clinical information and index test results were available to the assessors of the reference standard; 15.9%), item 18 (whether intended sample size and how it was determined were reported; 1.6%), item 19 (the flow of participants was indicated using a diagram; 17.5%), and item 25 (whether any adverse events from performing the index test or the reference standard were reported; 9.5%). By contrast, 12 items showed almost perfect adherence (> 95%), namely, item 1 (the study was identified as a study of diagnostic accuracy that used at least one measure of accuracy, item 2 (there was a structured summary of study design, methods, results, and conclusions), item 3 (scientific and clinical background, including the intended use and clinical role of the index test, was provided), item 4 (the study objectives and hypotheses were described), item 5 (it was indicated whether data collection was planned before or after the index test and reference standard were performed), item 6 (the eligibility criteria were listed), item 7 (the basis on which potentially eligible participants were identified was described), item 10a (the index test was described in sufficient detail to allow replication), item 12b (the definition of and rationale for test positivity cut-offs or result categories of the reference standard were described, with pre-specified cut-offs/result categories being distinguished from exploratory cut-offs/result categories), item 14 (methods for estimating or comparing measures of diagnostic accuracy were described), item 16 (how missing data on the index test and reference standard were handled was described), and item 27 (the implications for practice, including the intended use and clinical role of the index test, were described).
Of the 63 diagnostic test accuracy studies, there were 45 cohort-type accuracy studies and 18 case-control-type accuracy studies. The cohort- and case-control-type studies did not differ significantly in terms of total STARD score (20.2 ± 2.3 vs. 19.4 ± 1.7, p = 0.165) or QUADAS-2-related STARD score (11.6 ± 1.8 vs. 10.9 ± 1.5, p = 0.416).

Relationship between Adherence to STARD 2015 and Citation Rate

The mean ± SD number of times the included studies were cited between publication and March 2016 was 4 ± 4.6 (range 0–21). Multivariable linear regression analysis showed that exposure time correlated significantly with number of citations both when modelled with the total STARD score (partial correlation coefficient = 0.559, p < 0.001) and the QUADAS-2-related STARD score (partial correlation coefficient = 0.556, p < 0.001). When the effect of exposure time was accounted for, neither the total STARD score nor the QUADAS-2-related STARD score correlated significantly with number of citations (total score: partial correlation coefficient = 0.154, p = 0.232; QUADAS-2-related score: partial correlation coefficient = 0.143, p = 0.266).

DISCUSSION

This analysis indicates that the articles that reported diagnostic test accuracy studies and were published in KJR in the recent past were of moderate reporting quality: all articles reported more than 50% of the 27 STARD items while 20.6% articles reported more than 80% of the items. Moreover, 96.8% articles reported more than 50% of the 15 QUADAS-2-related STARD items while 31.7% articles reported more than 80%.
Several studies have previously investigated the quality of reporting diagnostic test accuracy studies in the radiology field. These studies employed the previous version of STARD (STARD 2003) and either assessed the quality of the reports at a particular single cross-sectional time (i.e., across various research areas or in a particular area) or analyzed the changes in quality over time before and after the release of STARD 2003 (9117980). These studies showed that the mean rate of adherence to STARD 2003 at specific time points ranged from 47.6% to 73.3% (9117980) and that the quality of the reports improved gradually, albeit mildly, after STARD 2003 was introduced (79). In the present study, the mean rate with which the diagnostic accuracy studies published in KJR in 2011–2015 adhered to STARD 2015 was 74%, which is at least consistent with the reported results and trend given that STARD 2015 has essentially all of the elements of STARD 2003 (as well as some new elements). This suggests that the journal had maintained its peer-review and editorial processes and reporting quality relatively well over the study period. Nevertheless, there is room for improvement, particularly in terms of the items that had a rather low adherence rate, namely, item 13b (whether clinical information and index test results were available to the assessors of the reference standard), item 18 (the intended sample size and how it was determined were described), item 19 (the flow of participants was indicated using a diagram), and item 25 (any adverse events from performing the index test or the reference standard were indicated).
The ultimate goal of reporting guidelines such as STARD is to ensure that the results of research studies are delivered accurately and clearly, thus improving their impact on patient care. Since it is difficult to measure the ultimate effect of an article, a journal or an adoption of reporting guidelines, the frequency with which an article is cited is often used as a surrogate variable. A recent study investigated the correlation between adherence to STARD 2003 and the citation rate (10). Unlike the present study, it found a weak positive correlation between these variables; however, the results were somewhat inconclusive because this correlation disappeared when the varying impact factor values of the different journals were accounted for (10). Our study lacked a between-journal effect; the fact that we did not detect a correlation between STARD adherence and citation rate may further support the notion that rate of adherence to STARD and citation rate do not really correlate. However, it remains possible that citation rate and journal impact factor do not accurately reflect the ultimate impact of an article or a journal, respectively (81). The ultimate effect of reporting guidelines on scientific publication may need to be examined more closely and thoroughly using alternative measures of article/journal impact.
This study had some limitations. First, because this study included articles published in a single journal, it may not be possible to directly generalize the results to other journals. However, the aim of this study design was to remove confounding caused by differences between journals. It would be worthwhile to also assess the relationship between STARD adherence and citation rate in other individual journals. Second, as mentioned above, the citation rate may not be the best variable for evaluating the ultimate impact or importance of an article. Better variables that allow a more robust analysis should be defined.
In conclusion, the degree of adherence to STARD 2015 was moderate for this particular journal, indicating that there is room for improvement. When adjusted for exposure time, the degree of adherence to STARD 2015 did not affect the citation rate.

Figures and Tables

Table 1

Rate of Adherence to Each STARD 2015 Item

kjr-17-706-i001
Item Adherence to STARD Items
All Articles (n = 63) Cohort-Type Accuracy Studies (n = 45) Case-Control-Type Accuracy Studies (n = 18)
Title or abstract
 1. Identification as study of diagnostic accuracy using at least one measure of accuracy (such as sensitivity, specificity, predictive values, or AUC) 96.8 (61) 97.8 (44) 94.4 (17)
Abstract
 2. Structured summary of study design, methods, results, and conclusions (for specific guidance, see STARD for Abstracts) 100 (63) 100 (45) 100 (18)
Introduction
 3. Scientific and clinical background, including intended use and clinical role of index test* 98.4 (62) 97.8 (44) 100 (18)
 4. Study objectives and hypotheses 96.8 (61) 100 (45) 100 (16)
Methods
 5. Whether data collection was planned before index test and reference standard were performed (prospective study) or after (retrospective study) 95.2 (60) 97.8 (44) 88.9 (16)
 6. Eligibility criteria* 95.2 (60) 97.8 (44) 88.9 (16)
 7. On what basis potentially eligible participants were identified (such as symptoms, results from previous tests, inclusion in registry)* 96.8 (61) 100 (45) 88.9 (16)
 8. Where and when potentially eligible participants were identified (setting, location, and dates)* 85.7 (54) 86.7 (39) 83.3 (15)
 9. Whether participants formed consecutive, random, or convenience series* 46.0 (29) 51.1 (23) 33.3 (6)
 10a. Index test, in sufficient detail to allow replication* 98.4 (62) 97.8 (44) 100 (18)
 10b. Reference standard, in sufficient detail to allow replication* 88.9 (56) 86.7 (39) 97.4 (17)
 11. Rationale for choosing reference standard (if alternatives exist)* 90.5 (57) 86.7 (39) 100 (18)
 12a. Definition of and rationale for test positivity cut-offs or result categories of index test, distinguishing pre-specified from exploratory* 84.1 (53) 86.7 (39) 77.8 (14)
 12b. Definition of and rationale for test positivity cut-offs or result categories of reference standard, distinguishing pre-specified from exploratory* 98.4 (62) 100 (45) 94.4 (17)
 13a. Whether clinical information and reference standard results were available to performers or readers of index test* 60.3 (38) 62.2 (28) 55.6 (10)
 13b. Whether clinical information and index test results were available to assessors of reference standard* 15.9 (10) 15.6 (7) 16.7 (3)
 14. Methods for estimating or comparing measures of diagnostic accuracy 100.0 (63) 100 (45) 100 (18)
 15. How indeterminate index test or reference standard results were handled* 90.5 (57) 93.3 (42) 83.3 (15)
 16. How missing data on index test and reference standard were handled* 98.4 (62) 100 (45) 94.4 (17)
 17. Any analyses of variability in diagnostic accuracy, distinguishing pre-specified from exploratory 28.6 (18) 26.7 (12) 33.3 (6)
 18. Intended sample size and how it was determined 1.6 (1) 2.2 (1) 0 (0)
Results
 19. Flow of participants, using diagram* 17.5 (11) 20 (9) 11.1 (2)
 20. Baseline demographic and clinical characteristics of participants* 90.5 (57) 88.9 (40) 94.4 (17)
 21a. Distribution of severity of disease in those with target condition* 73.0 (46) 71.1 (32) 77.8 (14)
 21b. Distribution of alternative diagnoses in those without target condition* 33.3 (21) 35.6 (16) 27.8 (5)
 22. Time interval and any clinical interventions between index test and reference standard* 58.7 (37) 64.4 (29) 44.4 (8)
 23. Cross-tabulation of index test results (or their distribution) by results of reference standard 84.1 (53) 82.2 (37) 88.9 (16)
 24. Estimates of diagnostic accuracy and their precision (such as 95% confidence intervals) 52.4 (33) 51.1 (23) 55.6 (10)
 25. Any adverse events from performing index test or reference standard 9.5 (6) 11.1 (5) 5.6 (1)
Discussion
 26. Study limitations, including sources of potential bias, statistical uncertainty, and generalizability 92.1 (58) 93.3 (42) 88.9 (16)
 27. Implications for practice, including intended use and clinical role of index test 96.8 (61) 95.6 (43) 100 (18)

Data are expressed as adherence rates in % and number of papers in parentheses. *Items (items 3, 6–13, 15, 16, and 19–22) that relate directly to Revised Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2). AUC = area under the curve, STARD = Standards for Reporting of Diagnostic Accuracy Studies

References

1. Equator Network, Essential Resources for Writing and Publishing Health Research. Web site. Accessed July 3, 2016. http://www.equatornetwork.org/.
2. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: The STARD Initiative. Radiology. 2003; 226:24–28.
3. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, et al. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ. 2015; 351:h5527.
4. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, et al. STARD 2015: An Updated List of Essential Items for Reporting Diagnostic Accuracy Studies. Radiology. 2015; 277:826–832.
5. Wikipedia. Open access journal. Web site. Accessed March 3, 2016. https://en.wikipedia.org/wiki/Open_access_journal.
6. Smith R. Peer review: reform or revolution? BMJ. 1997; 315:759–760.
7. Goldbeck-Wood S. Evidence on peer review-scientific quality control or smokescreen? BMJ. 1999; 318:44–45.
8. Van Raan AFJ. The use of bibliometric analysis in research performance assessment and monitoring of interdisciplinary scientific developments. Technikfolgenabschätzung. 2003; 1:20–29.
9. Smidt N, Rutjes AW, van der Windt DA, Ostelo RW, Reitsma JB, Bossuyt PM, et al. Quality of reporting of diagnostic accuracy studies. Radiology. 2005; 235:347–353.
10. Dilauro M, McInnes MD, Korevaar DA, van der Pol CB, Petrcich W, Walther S, et al. Is There an Association between STARD Statement Adherence and Citation Rate? Radiology. 2016; 280:62–67.
11. Wilczynski NL. Quality of reporting of diagnostic accuracy studies: no change since STARD statement publication--before-and-after study. Radiology. 2008; 248:817–823.
12. Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011; 155:529–536.
13. Rutjes AW, Reitsma JB, Vandenbroucke JP, Glas AS, Bossuyt PM. Case-control and two-gate designs in diagnostic accuracy studies. Clin Chem. 2005; 51:1335–1341.
14. Knottnerus JA, Muris JW. Assessment of the accuracy of diagnostic tests: the cross-sectional study. J Clin Epidemiol. 2003; 56:1118–1128.
15. Bossuyt PM, Leeflang MM. Developing Criteria for Including Studies. In : Deeks JJ, Bossuyt PM, Gatsonis C, editors. Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy Version 0.4 [updated September 2008]. Oxford: The Cochrane Collaboration;2008.
16. Song YS, Choi SH, Park CK, Yi KS, Lee WJ, Yun TJ, et al. True progression versus pseudoprogression in the treatment of glioblastomas: a comparison study of normalized cerebral blood volume and apparent diffusion coefficient by histogram analysis. Korean J Radiol. 2013; 14:662–672.
17. Moon JH, Park EA, Lee W, Yin YH, Chung JW, Park JH, et al. The diagnostic accuracy, image quality and radiation dose of 64-slice dual-source CT in daily practice: a single institution's experience. Korean J Radiol. 2011; 12:308–318.
18. Lee KH, Lee JM, Park JH, Kim JH, Park HS, Yu MH, et al. MR imaging in patients with suspected liver metastases: value of liver-specific contrast agent gadoxetic acid. Korean J Radiol. 2013; 14:894–904.
19. Lee DH, Lee JM, Klotz E, Kim SJ, Kim KW, Han JK, et al. Detection of recurrent hepatocellular carcinoma in cirrhotic liver after transcatheter arterial chemoembolization: value of quantitative color mapping of the arterial enhancement fraction of the liver. Korean J Radiol. 2013; 14:51–60.
20. Ko ES, Han BK, Kim SM, Ko EY, Jang M, Lyou CY, et al. Comparison of new and established full-field digital mammography systems in diagnostic performance. Korean J Radiol. 2013; 14:164–170.
21. Kim SA, Chang JM, Cho N, Yi A, Moon WK. Characterization of breast lesions: comparison of digital breast tomosynthesis and ultrasonography. Korean J Radiol. 2015; 16:229–238.
22. Kim JI, Kim YH, Lee KH, Kim SY, Lee YJ, Park YS, et al. Type-specific diagnosis and evaluation of longitudinal tumor extent of borrmann type IV gastric cancer: CT versus gastroscopy. Korean J Radiol. 2013; 14:597–606.
23. Kim J, Kim YH, Lee KH, Lee YJ, Park JH. Diagnostic performance of CT angiography in patients visiting emergency department with overt gastrointestinal bleeding. Korean J Radiol. 2015; 16:541–549.
24. Jung SI, Park HS, Yim Y, Jeon HJ, Yu MH, Kim YJ, et al. Added value of using a CT coronal reformation to diagnose adnexal torsion. Korean J Radiol. 2015; 16:835–845.
25. Jung SI, Park HS, Kim YJ, Jeon HJ. Multidetector computed tomography for the assessment of adnexal mass: is unenhanced CT scan necessary? Korean J Radiol. 2014; 15:72–79.
26. Iannicelli E, Di Renzo S, Ferri M, Pilozzi E, Di Girolamo M, Sapori A, et al. Accuracy of high-resolution MRI with lumen distention in rectal cancer staging and circumferential margin involvement prediction. Korean J Radiol. 2014; 15:37–44.
27. Chung SY, Park SH, Lee SS, Lee JH, Kim AY, Park SK, et al. Comparison between CT colonography and double-contrast barium enema for colonic evaluation in patients with renal insufficiency. Korean J Radiol. 2012; 13:290–299.
28. Bang SH, Lee JY, Woo H, Joo I, Lee ES, Han JK, et al. Differentiating between adenomyomatosis and gallbladder cancer: revisiting a comparative study of high-resolution ultrasound, multidetector CT, and MR imaging. Korean J Radiol. 2014; 15:226–234.
29. Zhang JJ, Liu T, Feng Y, Wu WF, Mou CY, Zhai LH. Diagnostic value of 64-slice dual-source CT coronary angiography in patients with atrial fibrillation: comparison with invasive coronary angiography. Korean J Radiol. 2011; 12:416–423.
30. Yu H, Cui JL, Cui SJ, Sun YC, Cui FZ. Differentiating benign from malignant bone tumors using fluid-fluid level features on magnetic resonance imaging. Korean J Radiol. 2014; 15:757–763.
31. Yoon SH, Goo JM, Jung J, Hong H, Park EA, Lee CH, et al. Computer-aided classification of visual ventilation patterns in patients with chronic obstructive pulmonary disease at two-phase xenon-enhanced CT. Korean J Radiol. 2014; 15:386–396.
32. Yoon HJ, Chung MJ, Hwang HS, Moon JW, Lee KS. Adaptive statistical iterative reconstruction-applied ultra-low-dose CT with radiography-comparable radiation dose: usefulness for lung nodule detection. Korean J Radiol. 2015; 16:1132–1141.
33. Yoo JY, Chung MJ, Choi B, Jung HN, Koo JH, Bae YA, et al. Digital tomosynthesis for PNS evaluation: comparisons of patient exposure and image quality with plain radiography. Korean J Radiol. 2012; 13:136–143.
34. Yi J, Lee EH, Kwak JJ, Cha JG, Jung SH. Retrieval rate and accuracy of ultrasound-guided 14-G semi-automated core needle biopsy of breast microcalcifications. Korean J Radiol. 2014; 15:12–19.
35. Wu EH, Chen YL, Wu YM, Huang YT, Wong HF, Ng SH. CT-guided core needle biopsy of deep suprahyoid head and neck lesions. Korean J Radiol. 2013; 14:299–306.
36. Wu CH, Huang CC, Wang LJ, Wong YC, Wang CJ, Lo WC, et al. Value of CT in the discrimination of fatal from non-fatal stercoral colitis. Korean J Radiol. 2012; 13:283–289.
37. Woo S, Kim SY, Cho JY, Kim SH. Shear wave elastography for detection of prostate cancer: a preliminary study. Korean J Radiol. 2014; 15:346–355.
38. Ucar M, Guryildirim M, Tokgoz N, Kilic K, Borcek A, Oner Y, et al. Evaluation of aqueductal patency in patients with hydrocephalus: three-dimensional high-sampling-efficiency technique (SPACE) versus two-dimensional turbo spin echo at 3 Tesla. Korean J Radiol. 2014; 15:827–835.
39. Tong T, Zhenwei Y, Xiaoyuan F. Transient ischemic attack and stroke can be differentiated by analyzing the diffusion tensor imaging. Korean J Radiol. 2011; 12:280–288.
40. Tomozawa Y, Inaba Y, Yamaura H, Sato Y, Kato M, Kanamoto T, et al. Clinical value of CT-guided needle biopsy for retroperitoneal lesions. Korean J Radiol. 2011; 12:351–357.
41. Sohn CH, Lee HP, Park JB, Chang HW, Kim E, Kim E, et al. Imaging findings of brain death on 3-tesla MRI. Korean J Radiol. 2012; 13:541–549.
42. Shin JE, Choi HJ, Kim MH, Cho KS. The serum CA-125 concentration data assists in evaluating CT imaging information when used to differentiate borderline ovarian tumor from malignant epithelial ovarian tumors. Korean J Radiol. 2011; 12:456–462.
43. Ryoo I, Suh S, Lee YH, Seo HS, Seol HY. Comparison of ultrasonographic findings of biopsy-proven tuberculous lymphadenitis and Kikuchi disease. Korean J Radiol. 2015; 16:767–775.
44. Rief M, Stenzel F, Kranz A, Schlattmann P, Dewey M. Time efficiency and diagnostic accuracy of new automated myocardial perfusion analysis software in 320-row CT cardiac imaging. Korean J Radiol. 2013; 14:21–29.
45. Park SO, Seo JB, Kim N, Lee YK, Lee J, Kim DS. Comparison of usual interstitial pneumonia and nonspecific interstitial pneumonia: quantification of disease severity and discrimination between two diseases on HRCT using a texture-based automated system. Korean J Radiol. 2011; 12:297–307.
46. Park B, Kim HK, Choi YS, Kim J, Zo JI, Choi JY, et al. Prediction of pathologic grade and prognosis in mucoepidermoid carcinoma of the lung using 18F-FDG PET/CT. Korean J Radiol. 2015; 16:929–935.
47. Niu XK, Bhetuwal A, Yang HF. CT-guided core needle biopsy of pleural lesions: evaluating diagnostic yield and associated complications. Korean J Radiol. 2015; 16:206–212.
48. Lyou CY, Cho N, Kim SM, Jang M, Park JS, Baek SY, et al. Computer-aided evaluation of breast MRI for the residual tumor extent and response monitoring in breast cancer patients receiving neoadjuvant chemotherapy. Korean J Radiol. 2011; 12:34–43.
49. Lv P, Mahyoub R, Lin X, Chen K, Chai W, Xie J. Differentiating pancreatic ductal adenocarcinoma from pancreatic serous cystadenoma, mucinous cystadenoma, and a pseudocyst with detailed analysis of cystic features on CT scans: a preliminary study. Korean J Radiol. 2011; 12:187–195.
50. Luczynńska E, Heinze-Paluchowska S, Dyczek S, Blecharz P, Rys J, Reinfuss M. Contrast-enhanced spectral mammography: comparison with conventional mammography and histopathology in 152 women. Korean J Radiol. 2014; 15:689–696.
51. Lu Q, Huang BJ, Wang WP, Li CX, Xue LY. Qualitative and quantitative analysis with contrast-enhanced ultrasonography: diagnosis value in hypoechoic renal angiomyolipoma. Korean J Radiol. 2015; 16:334–341.
52. Liu X, Peng W, Zhou L, Wang H. Biexponential apparent diffusion coefficients values in the prostate: comparison among normal tissue, prostate cancer, benign prostatic hyperplasia and prostatitis. Korean J Radiol. 2013; 14:222–232.
53. Lim HJ, Chung MJ, Lee G, Yie M, Shin KE, Moon JW, et al. Interpretation of digital chest radiographs: comparison of light emitting diode versus cold cathode fluorescent lamp backlit monitors. Korean J Radiol. 2013; 14:968–976.
54. Lee YH, Kim DW, In HS, Park JS, Kim SH, Eom JW, et al. Differentiation between benign and malignant solid thyroid nodules using an US classification system. Korean J Radiol. 2011; 12:559–567.
55. Lee SJ, Lee WW, Kim SE. Bone positron emission tomography with or without CT is more accurate than bone scan for detection of bone metastasis. Korean J Radiol. 2013; 14:510–519.
56. Lee S, Lee YH, Chung TS, Jeong EK, Kim S, Yoo YH, et al. Accuracy of diffusion tensor imaging for diagnosing cervical spondylotic myelopathy in patients showing spinal cord compression. Korean J Radiol. 2015; 16:1303–1312.
57. Lee MH, Kim SH, Kim H, Lee MW, Lee WJ. Differentiating focal eosinophilic infiltration from metastasis in the liver with gadoxetic acid-enhanced magnetic resonance imaging. Korean J Radiol. 2011; 12:439–449.
58. Lee KH, Goo JM, Park CM, Lee HJ, Jin KN. Computer-aided detection of malignant lung nodules on chest radiographs: effect on observers' performance. Korean J Radiol. 2012; 13:564–571.
59. Lee JH, Yoon YC, Jee S, Kwon JW, Cha JG, Yoo JC. Comparison of three-dimensional isotropic and two-dimensional conventional indirect MR arthrography for the diagnosis of rotator cuff tears. Korean J Radiol. 2014; 15:771–780.
60. Lee JH, Byun JH, Kim JH, Lee SS, Kim HJ, Lee MG. Solid pancreatic tumors with unilocular cyst-like appearance on CT: differentiation from unilocular cystic tumors using CT. Korean J Radiol. 2014; 15:704–711.
61. Lee JE, Lee JM, Lee KB, Yoon JH, Shin CI, Han JK, et al. Noninvasive assessment of hepatic fibrosis in patients with chronic hepatitis B viral infection using magnetic resonance elastography. Korean J Radiol. 2014; 15:210–217.
62. Lee EK, Choi SH, Yun TJ, Kang KM, Kim TM, Lee SH, et al. Prediction of response to concurrent chemoradiotherapy with temozolomide in glioblastoma: application of immediate post-operative dynamic susceptibility contrast and diffusion-weighted MR imaging. Korean J Radiol. 2015; 16:1341–1348.
63. Koo JH, Kim CK, Choi D, Park BK, Kwon GY, Kim B. Diffusion-weighted magnetic resonance imaging for the evaluation of prostate cancer: optimal B value at 3T. Korean J Radiol. 2013; 14:61–69.
64. Kim YP, Kannengiesser S, Paek MY, Kim S, Chung TS, Yoo YH, et al. Differentiation between focal malignant marrow-replacing lesions and benign red marrow deposition of the spine with T2*-corrected fat-signal fraction map using a three-echo volume interpolated breath-hold gradient echo Dixon sequence. Korean J Radiol. 2014; 15:781–791.
65. Kim SY, Moon SK, Jung DC, Hwang SI, Sung CK, Cho JY, et al. Pre-operative prediction of advanced prostatic cancer using clinical decision support systems: accuracy comparison between support vector machine and artificial neural network. Korean J Radiol. 2011; 12:588–594.
66. Kim SH, Kang BJ, Choi BG, Choi JJ, Lee JH, Song BJ, et al. Radiologists' performance for detecting lesions and the interobserver variability of automated whole breast ultrasound. Korean J Radiol. 2013; 14:154–163.
67. Kim MY, Cho N, Yi A, Koo HR, Yun BL, Moon WK. Sonoelastography in distinguishing benign from malignant complex breast mass and making the decision to biopsy. Korean J Radiol. 2013; 14:559–567.
68. Kim JE, Lee JY, Bae KS, Han JK, Choi BI. Acoustic radiation force impulse elastography for focal hepatic tumors: usefulness for differentiating hemangiomas from malignant tumors. Korean J Radiol. 2013; 14:743–753.
69. Kim HJ, Lee SH, Kang CH, Ryu JA, Shin MJ, Cho KJ, et al. Evaluation of the chondromalacia patella using a microscopy coil: comparison of the two-dimensional fast spin echo techniques and the three-dimensional fast field echo techniques. Korean J Radiol. 2011; 12:78–88.
70. Kim DW, Jung SL, Kim J, Ryu JH, Sung JY, Lim HK. Comparison between ultrasonography and computed tomography for detecting the pyramidal lobe of the thyroid gland: a prospective multicenter study. Korean J Radiol. 2015; 16:402–409.
71. Kang KM, Choi SI, Chun EJ, Kim JA, Youn TJ, Choi DJ. Coronary vasospastic angina: assessment by multidetector CT coronary angiography. Korean J Radiol. 2012; 13:27–33.
72. Kang KA, Kim YK, Kim E, Jeong WK, Choi D, Lee WJ, et al. T2-weighted liver MRI using the MultiVane technique at 3T: comparison with conventional T2-weighted MRI. Korean J Radiol. 2015; 16:1038–1046.
73. Jeh SK, Kim SH, Kang BJ. Comparison of the diagnostic performance of response evaluation criteria in solid tumor 1.0 with response evaluation criteria in solid tumor 1.1 on MRI in advanced breast cancer response evaluation to neoadjuvant chemotherapy. Korean J Radiol. 2013; 14:13–20.
74. Hur S, Kim HC, Chung JW, Kim MU, Kim JD, Kim GM, et al. Hepatocellular carcinomas smaller than 4 cm supplied by the intercostal artery: can we predict which intercostal artery supplies the tumor? Korean J Radiol. 2011; 12:693–699.
75. Chun KY, Choi YS, Lee SH, Kim JS, Young KW, Jeong MS, et al. Deltoid ligament and tibiofibular syndesmosis injury in chronic lateral ankle instability: magnetic resonance imaging evaluation at 3t and comparison with arthroscopy. Korean J Radiol. 2015; 16:1096–1103.
76. Choi HS, Kim AH, Ahn SS, Shin NY, Kim J, Lee SK. Glioma grading capability: comparisons among parameters from dynamic contrast-enhanced MRI and ADC value on DWI. Korean J Radiol. 2013; 14:487–492.
77. Cheung JY, Kim Y, Shim SS, Lim SM. Combined fluoroscopy- and CT-guided transthoracic needle biopsy using a C-arm cone-beam CT system: comparison with fluoroscopy-guided biopsy. Korean J Radiol. 2011; 12:89–96.
78. Ahn HS, Kim SM, Jang M, Yun BL, Kim B, Ko ES, et al. A new full-field digital mammography system with and without the use of an advanced post-processing algorithm: comparison of image quality and diagnostic performance. Korean J Radiol. 2014; 15:305–312.
79. Korevaar DA, Wang J, van Enst WA, Leeflang MM, Hooft L, Smidt N, et al. Reporting diagnostic accuracy studies: some improvements after 10 years of STARD. Radiology. 2015; 274:781–789.
80. Walther S, Schueler S, Tackmann R, Schuetz GM, Schlattmann P, Dewey M. Compliance with STARD checklist among studies of coronary CT angiography: systematic review. Radiology. 2014; 271:74–86.
81. Not-so-deep impact. Nature. 2005; 435:1003–1004.
TOOLS
Similar articles