PDF
ePub
Citation
Print
INTRODUCTION
Picture archiving and communication systems (PACS) have come to play an important role in radiological practice (1). In the interpretation of medical images, liquid crystal display (LCD) monitors have been replacing cathode ray tube (CRT) monitors due to their many advantages and nearly all workstation displays now use LCD panels (2).
Conventional LCD monitors with a cold cathode fluorescent lamp (CCFL) backlight have been reported to have problems with luminance and unevenness (3). In order to overcome these deficiencies, light emitting diode (LED) backlight matrices have recently been used for new types of LCD monitors in lieu of CCFLs (4). LEDs have better performance characteristics than CCFLs in terms of the voltage required for operation, frequency, reliability, temperature, and image quality (5). LEDs are even environment-friendly (mercury-free).
Light emitting diode monitors are being increasingly used for non-medical purposes for these reasons. They are not, however, commonly used in the medical field, and the overall performance of LED monitors has not been evaluated systematically in the field of diagnostic radiology. The purpose of our study was to compare the diagnostic performance of LED backlit monitors and CCFL monitors for the interpretation of digital chest radiographs.
MATERIALS AND METHODS
This study was designed as a multi-observer, multiple abnormalities observer performance receiver operating characteristic (ROC) study, in which observer performance was measured for the two types of monitors. Our institutional review board did not require its approval or informed consent for this study.
Case Selection
We selected postero-anterior chest radiographs that were obtained in asymptomatic patients that had undergone routine low dose chest computed tomography (CT) for lung cancer screening between January 1, 2010, and December 31, 2011. The abnormalities of interest were nodules and interstitial lung disease. Cases that were positive for these abnormalities were identified by a retrospective search of the hospitalls radiology information system database. Cases with proven lung nodules or interstitial lung disease were enrolled on the basis of the CT findings. The chest radiographs were obtained no more than one week before or after a CT scanning. Selection criteria for the nodules included the following: size < 3 cm in diameter, and solid nodules or focal air space opacities that were seen as nodules on chest radiographs. The cases of interstitial lung disease were selected on the basis of the CT findings of ground-glass opacities, reticular opacities, and/or honeycombing. Normal chest radiographs were also selected on the basis of the CT scans.
Two chest radiologists (with 19 and four years of experience in chest radiography respectively) viewed and categorized each radiograph according to the presence or absence of abnormalities. Radiographs with image artifacts such as respiratory motion, an inadequate field of view, or poor positioning were excluded. All decisions were made by consensus.
There were 43 occurrences of nodules (Fig. 1A, B) and 25 occurrences of interstitial disease (Fig. 1C, D) on a total of 65 radiographs. Three images showed both abnormalities. We also selected 65 radiographs that were negative for either abnormality that were matched according to sex and age with the positive population (Fig. 1E, F). A total of 130 postero-anterior chest radiographs were collected for this study.
Image Acquisition and Display
The postero-anterior chest radiographs were obtained using a CsI and amorphous silicon flat panel system (GE Definium 8000, GE Healthcare, Milwaukee, WI, USA). The image data had a matrix of 2022 × 2022, and a gray-scale depth of 14 bits. The soft copy images were displayed with default post-processing transformation for presentation into the DICOM image file and sent to a PACS server for distribution to display workstations.
The images were evaluated on a 3-megapixel LCD color monitor (CCL352i; Totoku, Tokyo, Japan) and a 3.5-megapixel LED color monitor (SB970; Samsung, Seoul, Korea). The LCD monitor had a matrix of 2048 × 1536 pixels with a 10-bit look-up table and a maximum brightness level of 600 cd/m2. The LED monitor had a matrix of 2560 × 1440 pixels with a 16-bit look-up table and a maximum brightness of 300 cd/m2. We used a Digital Imaging and Communications in Medicine grayscale standard display function, or DICOM GSDF, for calibration of the monitors.
Study Performance
The study consisted of two sessions in which each of the two monitors was used. Eight observers read and rated each of the 130 cases during both sessions. The image reading sessions were conducted in the reading room of the radiology department. Prior to the study, all observers were instructed on the definition of the abnormalities and the presence of normal cases, and were trained in how to use the scoring system. The observers were blinded to the proportion of images containing abnormalities. Discussion for consensus about negative and positive cases was not permitted. The viewing sessions as well as the order in which the images were presented were randomized, and there was at least a one-week interval between each reading session in order to reduce learning bias. There was no limit on the reading time. To simulate daily reading practice, observers were allowed to adjust the window width and level of the images.
For the ROC analysis, observers were asked to determine the presence of abnormalities using a continuous rating scale from 0-10. The rating scale represented each observer's confidence level regarding the presence of nodules and interstitial lung disease. Zero indicated that the abnormality was not present with absolute certainty, and 10 indicated that the abnormality was present with absolute certainty. Ratings between 0 and 10 were interpreted as intermediate levels of confidence (1-4, probable negative; 5, neutral; 6-9, probable positive).
The reading time was recorded for each observer during all sessions. After finishing each session, the observers were asked to determine their subjective visual fatigue score and heating sense using a point-scale ranging from 0 (no feeling of fatigue or no heating sense) to 10 (extreme feeling of fatigue or heating sense).
To measure radiant heat from the monitors, we compared the temperature rise before and after 1 hour of use for each type of monitor. The temperature rise was measured using a 1 m3 box with hermetic sealing. In order to determine the brightness of the monitors, we used a portable luminance meter at a distance of 1 m directly in front of the monitors.
Observers
Four board-certified chest radiologists (all with four years of experience in chest radiography) and four residents (with one, two, three years and one month of experience in chest radiography, respectively), each experienced in using PACS, independently reviewed all digital radiographs. The radiologists who participated in the preparation of the study and case selection were not among the eight observers. Among the observers, the level of experience with chest radiography ranged from one month to five years (mean, 2.83 years).
Data and Statistical Analysis
A total of 2080 observations (130 sets × 2 detector systems × 8 observers) were evaluated. For each reader, the detection performance for nodules and interstitial lung disease was assessed using the two monitors.
Using the CT results as references, we categorized the observers' scores for all 130 cases. All scores were reclassified as true positive, false positive, false negative, and true negative. From these data, we calculated detection sensitivity, exclusion specificity, and overall accuracy.
In this study, we used a multi-reader, multi-case receiver operating characteristic approach (6, 7). Statistical significance was reported using a 95% confidence interval for the mean differences in area under curve (AUC) values for observer performance. Sensitivity, specificity and accuracy in the detection of nodule or interstitial lung marking (ILD) were compared between monitor types for all readers and for either subgroup of readers (the four board-certified chest radiologists were referred to as group B and the four residents were referred to as group R) using the paired t test. Reading times, fatigue scores, and sense of heat were compared using Wilcoxon signed-rank tests. All analyses were performed using SPSS software for Windows version 17.0 (SPSS Inc., Chicago, IL, USA). P-values < 0.05 were considered to be statistically significant.
RESULTS
Performance Characteristics of the Two Monitors
Measured brightness was 291 cd/m2 for the LED and 354 cd/m2 for the CCFL monitors. The temperature elevation measured after 1 hour of use was 6.7℃ for the LED and 12.4℃ for the CCFL monitors.
Observer Performance
Tables 1, 2 summarize the observer performances for the detection and diagnosis of lesions on the LCD monitors with CCFL versus LED backlights including the sensitivity, specificity, and accuracy. When all readers were considered, there was a statistically significant difference in sensitivity for ILD (p = 0.025, the LED monitor was superior to the CCFL monitor). For the rest, however, there was no significant difference in performance in detecting nodules or ILD for the two types of monitors studied. However, when board-certified chest radiologists (observer 1-4) and residents (observer 5-8) were separately analyzed, sensitivity and accuracy for ILD were significantly better in the LED monitor with the board certified radiologists (p = 0.016 and 0.013, respectively). With the residents, there was no difference in the detection performance of nodules or ILD between the two types of monitors.
Receiver operating characteristic curves for the eight observers for nodules and ILD are shown in Figure 2. Table 3 and Figure 3 depict the AUC for each observer and the average AUC for nodules and interstitial lung disease using the two types of LCD monitors. AUCs for nodule detection were 0.721 ± 0.072 (mean ± SD) and 0.764 ± 0.098 for LED and CCFL (p = 0.173), respectively, whereas those for ILD were 0.871 ± 0.073 and 0.844 ± 0.068 (p = 0.145), respectively.
Fatigue Score and Heating Sense
Table 4 shows that fatigue score and sense of heat ranged from 0-8 for the two monitors. There was no significant difference between the mean fatigue score for the color LCD monitor with the CCFL backlight and that of the color LED monitor (p = 0.102).
The mean sense of heat score for the CCFL monitor (4.4); however, this was significantly higher than that of the LED monitor (2.6) (p = 0.024). All observers except observer 7 agreed that they felt less heat after using the LED monitor.
Interpretation Time
The total interpretation time averaged 61.5 minutes (range: 46-98 minutes) for the LCD monitor with the CCFL backlight and 68.8 minutes (range: 40-80 minutes) for the LCD monitor with the LED backlight. Interpretation times were not significantly different between the LED and CCFL monitors (p = 0.450).
DISCUSSION
High-quality images are required for the accurate interpretation of medical images by radiologists. For this reason, monitors must meet strict performance criteria (8). A new generation of LCD monitors with LED backlights have been developed and are currently being used in place of those with CCFL backlights in non-medical fields, given their advantages of higher contrast, lower power consumption, durability, and safety (they contain no hazardous material such as mercury) (9). These physical characteristics are indicators of overall system performance; however, no physical measurement correlated perfectly with the perceived diagnostic performance. Therefore, clinical performance cannot be predicted from the physical characteristics of the monitors themselves, and additional observer performance studies are needed to further evaluate this finding (10). To the best of our knowledge, this is the first study that focuses on diagnostic performance using LCD monitors with an LED backlight.
In the present study, we evaluated image quality and diagnostic performance using LCD monitors with conventional CCFL and LED backlight systems. Diagnostic performance was assessed in terms of the ability to detect nodules or increased interstitial markings on digital chest radiography.
Our results indicate that there are no significant differences in the detection of nodules and interstitial lung disease using LCD monitors with LED backlights versus conventional CCFL backlights. However, this study cannot conclude the equivalence of the two types of monitors. The observations from this study might be regarded as hypothesis-generating and warrant a future non-inferiority study.
In addition to resolution, luminance range, contrast, and durability should also be considered when choosing an appropriate monitor for medical imaging (9). The brightness of the monitor refers to the emitted luminous intensity on the screen (11), and is measured in candela per square meter (cd/m2 or nit). A higher cd/m2 or nit value indicates a higher onscreen brightness (11). Luminance is defined as the absolute quantity of radiation emitted from a given source of visible electromagnetic radiation (11). The higher luminance offered by medical displays results in better image quality, making subtle lesions easier to detect (12). The American College of Radiology (ACR) has specified that the luminance of a monitor to be used for the primary interpretation of digital radiographs should be at least 171 cd/m2 (50 foot-lamberts) (13). Although the measured brightness level was lower in LCD monitors with an LED backlight (291 cd/m2) than those with a CCFL backlight (354 cd/m2) in this study, the brightness of the LED monitor met the criteria recommended by the ACR and it is thought to be adequate for the interpretation of radiographic images. In terms of power consumption, LED monitors used less power (69 W in 100-220 V) than LCD monitors with a CCFL backlight (130 W in 100-220 V).
In this study, we evaluated the diagnostic performance of observers with variable levels of experience in chest radiology. Although we did not electronically measure how often observers manipulated the window settings in this study, all observers responded that they used this function in most of the cases. Window width and level adjustments are the most commonly used workstation tools in soft copy interpretation. Because most digital images have a much wider dynamic range than do monitors, limiting the adjustment of the window width and level of an image sacrifices an important advantage of using digital images (14).
There was no difference in reading time between the two monitors, with variations in reading time reflecting the individual propensity to interpret an image. The fatigue score was lower for LCD monitors with an LED backlight. This may be due to the lower level of brightness, which would be consistent with a previous report by Goo et al. (14) in that fatigue was greater at high levels of monitor luminance (100 fL) than low or middle levels (25, 50 fL). In the current study, we also determined the radiant heat of the monitors by measuring the temperature rise after 1 hour of use, and found a smaller change in temperature in the LED monitors. The sense of heat scored after each session by observers was also lower for the LED monitors.
There are several limitations to the current study. First, the detection performance was evaluated only by radiologists in the subdued ambient environment of the radiology reading room. Differences in diagnostic performance between the two monitors may change under other types of ambient illumination. Additionally, the performance of clinical colleagues (non-radiologists) may not be the same in brightly lit conditions. Thus, the effects of ambient illumination deserve further consideration. We also allowed adjustment of the window width and level of the images, and there were differences in the gray scale of each imaging system. This may also have affected observer performance, allowing for the detection of subtle lesions by enhancing the detailed contrast ratio (14). While this may be considered a limitation, we allowed the adjustment of the window width and level of the images in order to simulate daily clinical practice. A potential preference bias for each observer could not be eliminated because both monitor types could be readily identified. While color LCD monitors with two different backlight systems were compared in this study, it is known that the native brightness and contrast of monochrome displays are higher than those of color LCD displays (10). Further studies are therefore needed to evaluate the utility of monochrome LCD monitors with LED backlights. Lastly, Krupinski (15) suggested that medical-grade displays are generally more stable over time in terms of maintaining consistent levels of backlighting and thus luminance levels. The commercial off-the-shelf color display may degrade enough to negatively impact diagnostic and visual search performance. Further studies should compare diagnostic performance using both types of displays that have been set to luminance levels that would be expected after years of use.
In conclusion, use of LED monitors resulted in less radiant heat compared with conventional LCD monitors. In addition, observer detection of pulmonary nodules and interstitial lung disease on chest radiographs was not significantly different between LCD and LED monitors except for sensitivity of ILD in our exploratory study. To the best of our knowledge, no research has been carried out in this area and our results warrant further investigation of non-inferiority of LED monitors in the future. Also, further observer studies of other anatomical sites, multiple medical disorders, bright ambient conditions, and different degrees of subtlety of lung nodules may be necessary for establishing appropriate criteria for medical-grade LED monitors.
XML Download