PDF
ePub
Citation
Print
I. Introduction
During the last half century, in the development and the diversification of medical field scientific technology, hospital information systems (HIS) has provided new knowledge needed for work safety and precision using IT, and has thus received attention as an important tool for enhancing the quality of health and medical services [1].
The rapid proliferation and development of the HIS has increased the standards of quality improvement of medical service and the optimization of hospital management. For the enhancement of quality and effectiveness of work process related to medical use, clinical decision support systems (CDSS) linked to computerized provider order entry (CPOE) system has come into use. However, compared to the emergence of electronic medical record (EMR), CDSS is still in an early stage as its use is insufficient, and its development and utilization for the prescription and dispensing of medicines are limited.
Through many studies, CPOE integrated with CDSS has reduced adverse drug reaction cases and medication errors, and this suggests that drug use safety has been enhanced [2-5]. In Korea, several studies have analyzed prescription changes before and after the application of CDSS for medication. Lee et al. [6] found that there was a reduction in medication errors after using CDSS for medication. However, for it to be effective in reducing medication errors, key users of CDSS need to be satisfied with it. Accordingly, user satisfaction is a critical success factor (CSF) for CDSS in many studies [2,4,7,8]. DeLone and McLean [9,10] subdivided success measures of information systems into six distinct categories: 1) system quality, 2) information quality, 3) user satisfaction, 4) usage, 5) individual impact, and 6) organizational impact. Van Der Meijden et al. [11] examined the determinants of success of inpatient clinical information systems according to the DeLone and McLean [9,10] framework. Park et al. [12] have analyzed the performance of the CDSS for medication in 38 hospitals by using the DeLone and McLean [9,10] success model of the information system and found that both system quality and information quality significantly influenced user satisfaction. However, these findings were based on only responses from pharmacists, excluding doctors who are responsible for maintaining drug safety.
This study aimed to investigate satisfaction among all users (doctors, nurses, and pharmacists) with CDSS and its effects on enhancing drug safety. Specifically, this study identified the factors influencing user satisfaction in terms of system satisfaction and information satisfaction, compared degree of user satisfaction among professional groups, and examined the areas for further improvement in the CDSS.
II. Methods
1. Subjects
Satisfaction of CDSS use for medication was investigated through a survey involving doctors, nurses, and pharmacists who worked in Pusan National University Hospital which has 1,170 beds. The survey was distributed directly during a visit to the hospital from September 7 through September 27, 2011. Two hundred and eighteen respondents were collected based on convenient sample and analyzed in the study. The study hospital did not require mandatory use of EMR for doctors and therefore most of doctors in the sample were young doctors.
2. Model and Statistical Analysis
Based on the DeLone and McLean [9,10] framework, we evaluated the user satisfaction of the CDSS by using three success measures (system quality, information quality, and support factor) as independent variables, and sum of two satisfaction measures (system satisfaction and information satisfaction) as dependent variables for a multiple regression analysis (Figure 1). Four measures were used for evaluating system quality: ease of system use, ease of understanding results, ease of understanding terminology, and usefulness of system user interface (UI). Five measures were used for evaluating information quality: information accuracy, timeliness, reliability, up-to-datedness, and decision support. In addition, three supporting factors were used for evaluating organizational supports: user training support, top management support, and related department support. All measures were originally coded in 5-point scale but converted into 3-point scale during the analyses in order to reduce empty cells.
In addition, associations between three success measures and system overall satisfaction was analyzed based on cross-tabulation. We also analyzed the associations between three success measures and drug safety improvement. However, chi-square test was not carried out because of small cell size in many items. In addition, each of satisfaction measures was compared for three professional groups by using analysis of variance (ANOVA). The SAS 9.1 package (SAS Institute Inc., Cary, NC, USA) was used for statistical analysis of the data.
III. Results
1. Characteristics of the Subjects
The general characteristics of the survey respondents are shown in Table 1. The total number of respondents in the survey was 218; 34% were males and 66% were females. For age group distribution, 211 of them were under 40 years, comprising the largest age group. The occupational composition was 47% (doctors), 46% (nurses), and 7% (pharmacists). The respondents were mostly from the internal medicine and surgery departments, 42% and 40%, respectively. Finally, 59.3% of the respondents had 2-5 years of hospital work experience.
2. The Effect of Success Measures on User Satisfaction
Among the success measures, information reliability, decision supporting factor, and related department support factor were significant factors influencing user satisfaction as shown in Table 2. The R-square of this model was 0.65.
3. Analysis of Association between Success Measures and User Satisfaction
To analyze the association between success measures and user satisfaction in detail, we selected overall system satisfaction from system satisfaction measures and drug satisfaction improvement from information satisfaction measures.
1) Association between success measures and overall system satisfaction
As seen in Table 3, positive responses (Yes) in overall system satisfaction had consistently higher positive responses (Yes) in all success measures. Specifically, the highest positive response was information reliability (82.9%), followed by usefulness of system user interface (81.6%) and information timeliness (77.6%) in order. Compared with system quality and information quality measures, however, all support factors had lower percentage of positive satisfaction responses. The highest positive response on support factors was the related department support (48.0%), followed by user training support (44.8%).
2) Association between success measures and drug safety improvement
As seen in Table 4, positive responses (Yes) in drug safety improvement also had consistently higher positive responses (Yes) in all success measures. While overall trends were similar between overall system satisfaction and drug safety improvement, orders of importance were different for two measures. Specifically, the highest positive response was decision supporting (82.0%), followed by usefulness of system user interface (75.3%) and information reliability (73.0%) in order. Compared with system quality and information quality measures, however, all support factors also had lower percentage of positive satisfaction responses. The highest positive response on support factors was user training support (43.8%), followed by the related department support (43.2%).
4. Comparison of Satisfaction by Professional Group
There are differences in user satisfaction by professional group among doctors, nurses, and pharmacists, who use the CDSS (Table 5). Nurses had the highest level of satisfaction, followed by pharmacists and doctors. Doctors and nurses had the highest satisfaction level in drug safety improvement, and pharmacists had the highest satisfaction level in information usefulness. When each factor of satisfaction level was compared, doctors were less satisfied in overall system satisfaction, task satisfaction, and information usefulness factors than nurses and pharmacists.
5. Comparison of Success Factors Influencing Drug Safety Improvement
We allowed for multiple responses to compare the responses on success factors influencing drug safety improvement. One hundred fifty-one (69.3%) respondents stated that the timely provision of drug information (product information, drug identification information, drug ingredient information, medication guide, national health insurance guide, etc.) was the function with the most support in improving drug safety (Table 6). Fifty-nine (27.1%) respondents said that the decision support function of CDSS (dose alert, drug interactions alert, duplication alert, and drug allergy alert) helped to improve drug safety, 35 (16.1%) respondents answered that medication support tools (medication instruction output tool, identification report output tool, administration tool) helped in drug safety improvement, and 5 (2.3%) respondents answered that checking function for drug utilization review (DUR) requirements by Health Insurance Review Agency (HIRA) also helped drug safety improvement.
Regardless of their profession, all respondents stated that drug information providing function provided the most support in drug safety improvement. The second most answered function was decision support function, of which 26.6% of doctors and 25.3% of nurses responded that it helped to improve drug safety. For the pharmacist group in which the decision support function does not apply, the second chosen function to drug safety improvement was medication support tools.
IV. Discussion
This study examined the effects of quality measures of CDSS (system quality and information quality) and its support factors (top management, department support, and user training) on satisfaction factors (user satisfaction and information satisfaction) by using the information system success model of DeLone and McLean [9,10]. We found that two information quality factors (information reliability and decision supporting capability) and one supporting factor (departmental support) were significant factors influencing user satisfaction.
When user satisfaction was separately analyzed by system satisfaction and information satisfaction, ease of use and decision support capability significantly influenced both user satisfaction factors. Ease of use was also a significant factor influencing user satisfaction in studies by Park et al. [12] and Kulkarni et al. [13]. This finding supported the ultimate goal of CDSS for medication, which provides information to help make a better decision in prescribing medicine.
We also analyzed the association between success measures and overall system quality and found that positive responses (Yes) in overall system satisfaction had consistently higher positive responses (Yes) in all success measures. Compared with system quality and information quality measures, however, all support factors had lower percentage of positive satisfaction responses. This shows an importance of management supports in improving user satisfaction suggested by the previous studies [9-11].
Degree of user satisfaction was the highest for nurses, followed by pharmacists and doctors. Low satisfaction level for doctors may be explained by the fact that the CDSS was primarily designed from administrative perspective focusing on reducing medication errors rather than doctors' clinical decision-aid such clinical practice guideline for prescription. Primary concern in information satisfaction also varied among professions. Drug safety was the highest for doctors and nurses, while usefulness of information was the highest for pharmacists. In addition, 69.3% of users responded that timely provision of drug information were the most important system functions to the CDSS in improving drug safety.
In conclusion, we found that user satisfaction with the capability of enhancing drug safety was reasonably high. However, real time safety checking function, which was a major concern for doctors, should be improved further in order to increase the satisfaction level of doctors and reduce medication errors. In addition, Our study showed that the dimensions of success defined by DeLone and McLean [9,10] for information management systems are applicable to the CDSS for medication, similar to a study on inpatient care information systems by Van Der Meijden et al. [11].
This study has a couple of limitations. First, the user satisfaction, which is the key indicator of CDSS evaluation, is a measurement of the perception on the system of users. Thus, there is a possibility of the indicator being subjective. Many studies have mentioned that quantifying measured factors is difficult when evaluating the outcomes of information systems. It is difficult to separate the effect of the information system objectively. However, presenting objective data, such as the cost-benefit analysis of information system [4], the real usage of detailed function of information system, and reduced time of decision-making, is needed.
Second, our findings may have external validity limitations. This study was conducted in only one hospital; as such, it is difficult to apply these results to other hospitals that may have different task process and characteristics. In addition, the study targeted one CDSS, so it is hard to generalize the results to user satisfaction on all CDSS. The environments of hospitals, such as task characteristics and size, are different from one another and they can be affected by exogenous variables deriving from circumstances of each hospital. For the results to be more applicable, systematic research of various hospital environments and systems will be needed.
XML Download