Role of Computerized Physician Order Entry Usability in the Reduction of Prescribing Errors

Hamid Reza Peikari; Mohamad Shanudin Zakaria; Norjaya M. Yasin; Mahmood Hussain Shah; Abdelbary Elhissi

doi:10.4258/hir.2013.19.2.93

Journal List > Healthc Inform Res > v.19(2) > 1075684

Go to TopGo to Top Go to BottomGo to Bottom

TOOLS

Peikari, Zakaria, Yasin, Shah, and Elhissi: Role of Computerized Physician Order Entry Usability in the Reduction of Prescribing Errors

Original Article

Healthcare Informatics Research 2013; 19(2): 93-101.

Published online: 30 June 2013

DOI: https://doi.org/10.4258/hir.2013.19.2.93

Role of Computerized Physician Order Entry Usability in the Reduction of Prescribing Errors

Hamid Reza Peikari, PhD¹, Mohamad Shanudin Zakaria, PhD¹, Norjaya M. Yasin, PhD¹, Mahmood Hussain Shah, PhD², Abdelbary Elhissi, BSc, PhD³

¹Graduate School of Business, Universiti Kebangsaan Malaysia, Bangi, Malaysia.

²Lancashire Business School, University of Central Lancashire, Preston, UK.

³School of Pharmacy and Biomedical Sciences, University of Central Lancashire, Preston, UK.

Corresponding Author: Hamid Reza Peikari, PhD. School of Pharmacy and Biomedical Sciences, Universiti Kebangsaan Malaysia, 43600 UKM Bangi, Selangor, Malaysia. Tel: +60-10-893-9485, Fax: +60-3-8921-3161, omid726@yahoo.com

Received 13 February 2013 Revised 1 June 2013 Accepted 5 June 2013

(open-access):

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Objectives

Some hospitals have implemented computerized physician order entry (CPOE) systems to reduce the medical error rates. However, research in this area has been very limited, especially regarding the impact of CPOE use on the reduction of prescribing errors. Moreover, the past studies have dealt with the overall impact of CPOE on the reduction of broadly termed "medical errors", and they have not specified which medical errors have been reduced by CPOE. Furthermore, the majority of the past research in this field has been either qualitative or has not used robust empirical techniques. This research examined the impacts of usability of CPOE systems on the reduction of doctors' prescribing errors.

Methods

One hundred and sixty-six questionnaires were used for quantitative data analyses. Since the data was not normally distributed, partial least square path modelling-as the second generation of multivariate data analyses-was applied to analyze data.

Results

It was found that the ease of use of the system and information quality can significantly reduce prescribing errors. Moreover, the user interface consistency and system error prevention have a significant positive impact on the perceived ease of use. More than 50% of the respondents believed that CPOE reduces the likelihood of drug allergy, drug interaction, and drug dosing errors thus improving patient safety.

Conclusions

Prescribing errors in terms of drug allergy, drug interaction, and drug dosing errors are reduced if the CPOE is not error-prone and easy to use, if the user interface is consistent, and if it provides quality information to doctors.

Keywords: Medical, Usability, Health Informatics, CPOE

I. Introduction

Medical errors in hospitals and clinics are not rare and may cause severe harm to patients [1]. While Bates et al. [2] reported that medical errors cause more than one million injuries in United States hospitals every year, Bosman [3] reported 1.7 medical errors per patient. Furthermore, medical errors are one of the top ten causes of mortality [4]. Blendon et al. [5] reported that the mortality rate due to medical errors is higher than the mortality rate as the result of accidents. Moreover, such errors impose huge costs on the health sector [6,7].

1. Problem Statement

Computerized physician order entry (CPOE) systems are one of the clinical information system (CIS) solutions implemented in hospitals to improve patient safety and reduce medical errors [8]. However, the results of the research in this area are mixed. While some studies have reported positive impacts on medical error reduction and patient safety [9,10], other studies have shown that these systems do not always reduce medical errors, and they may even increase medical errors and mortality rates [3,11]. Despite the importance of this issue, little knowledge is available about the factors leading to the reduction of medical errors by CPOE [12]. Indeed, the past research has mainly focussed on the overall impact of CPOE systems on the reduction of medical errors, which has recently motivated scholars to study the impact of CPOE design on the reduction of medical errors [13]. Hence, it is needed to study the CPOE features which lead to the reduction of medical errors.

Some previous studies "have examined evidence of the impact of CPOE on medication errors, but have used highly variable definitions of error" [14]. A major problem in past studies is that the measurement of error is often ambiguous, and more specifications are needed in this regard [14]. Moreover, in some studies, the distinction between prescription errors and other types of medical errors is not clear [14]. Thus, there is a need to study the impact of CPOE on specific prescribing errors rather than broadly termed medical errors.

There is another issue in this field regarding the research method used to study the impact of CPOE on positive care outcomes. It is argued that although some studies have reported that CPOE facilitates the reduction of errors, none of them have had sufficient rigor to show significant statistical findings [15]. Indeed, past attempts to measure the usability of information systems have had two shortcomings: they have either used qualitative research methods that restrict the generalization of the findings or they have not used a robust validated quantitative scale to assess usability [16]. In a recent study, Weir et al. [17] reviewed the quantitative studies that have researched the outcomes of CIS and concluded that "none of the reviews provided usable quantitative data" [17]. They suggested that future attempts should quantitatively evaluate the outcomes of CIS [17]. Therefore, a robust empirical study is needed to examine the impacts of CPOE usability on care outcomes, such as the reduction of errors. Besides, most of the past studies have collected data from a single hospital, which undermines the generalizability of their findings [14,18], or they have used insufficient sample size for their analyses, which reduces the strength of their analyses and findings [7,14,19]. Therefore, data should be collected from sufficient sample size in multiple sites to increase the generalizability of the findings.

2. Objectives and Contribution

The aim of this research was to rigorously and quantitatively examine the influence of the use of CPOE on reduction of the prescribing errors. More specifically, this research was intended to answer the following questions using the quantitative research method:

1) What is the influence of the quality of the information produced by the system on the reduction of medical staffs' prescribing errors?
2) Does the ease of use of a clinical information system lead to a reduction of users' prescribing errors?
3) Is the impact of user interface consistency and system error prevention features of CPOE on the reduction of prescribing errors, mediated by system ease of use positive and significant?

The results will improve our knowledge about usability attributes that can lead to the reduction of prescribing errors. Furthermore, since medical errors are one of the indexes of patient safety [20] and medical service quality [21], the results can ultimately assist us to improve patient safety and medical service quality by improving the CPOE usability features that significantly contribute to these outcomes.

Another contribution is that, since doctors are the second victims of medical errors due to emotional, legal and professional difficulties they face, assessing the extent to which CPOE can reduce their errors can contribute in a reduction of doctors' negative work-related experiences.

Moreover, since this field lacks robust empirical research [15-18], the reliability and validity of previous analyses and the generalization of their findings are limited. The qualitative studies in this area are also restricted by the subjective perceptions and evaluations of the researchers in data collection and analysis which restrict the generalization of their findings [22]. Since this study collects data from a methodologically sufficient sample size, and applies a systematic multivariate data analyses process suggested by Hair et al. [23], its analyses and findings are more reliable, valid, rigorous, and generalizable. Furthermore, the scale developed and validated by this study can be used for future studies.

3. Theories and Hypotheses

Although system usability has been studied in numerous research papers, scholars have various opinions on how to define and measure system usability. Usability is a broad term that includes a variety of parameters (for instance see Zhang et al. [4], Oztekin et al. [10,16], Maenpaa et al. [12], Corrao et al. [19], Joshi et al. [24], and Gonzalez et al. [22]). This implies that, despite the attention of researchers to study system usability, there is no consensus among researchers on how to measure usability. This research refers to system usability as the ease of use, information quality, consistency, and error prevention features of a system.

The theoretical basis for this research is the input-process-output model [25]. According to this model, to improve users' performance (such as providing error-free medical services) as the result of using the CIS, the quality of the system features should be improved. Hence, it is hypothesized that:

H1: The ease of use of a CPOE positively reduces doctors' prescribing errors.
H2: The information quality produced by a CPOE positively reduces doctors' medical errors.

Moreover, it seems that there are some relationships between usability features [26]. According to Davis [27], the determinants of system ease of use suggested in the technology acceptance model (TAM) should be found. According to TAM, the ease of use and usefulness of a technology are the determinants of users' acceptance of a technology. However, the model (i.e., TAM) does not provide any insight into the factors that lead to the perceived ease of use of a technology. This gap has been identified by Davis [27], who called for more research in this regard. Some authors [28-30] suggest that system features are the determinants of system ease of use. Hence, it is suggested that:

H3: The consistency of a CPOE system positively leads to its ease of use.
H4: The error prevention features of a CPOE system positively predicts its ease of use.

II. Methods

This research employed a quantitative method using a self-administered survey. The target population included doctors who had at least 3 months of experience with CPOE systems. Except the demographic questions, other questions were formatted in a 5-point Likert scale from 1 (strongly disagree) to 5 (strongly agree).

1. Self-Report Scale

Using a self-report survey for this research is methodologically acceptable since it has been recognized and used by many researchers to measure the features and outcomes of information systems [7,10,31]. Furthermore, self-report scales can provide an important indicator of an information system's individual and organizational outcomes [32]. Conway and Lance [33] reviewed some previously published studies about self-report scales and concluded that using other methods of measurement such as secondary data and other-report measures are not methodologically or analytically superior to self-report measurements. Similarly, Fleenor et al. [34] stated that other-report scales are not necessarily more accurate than self-report measures. Furthermore, self-report scales may have the advantage of self-awareness of the participants [34]. Besides the quantitative studies, qualitative studies (such as those, using think aloud method) have also used self-report measures to study system features and system outcomes. These imply that other methods of measurement, such as using secondary data, are not methodologically or analytically superior to self-report scales. Therefore, using self-report evaluations of health professionals in this research provides reliable and valid insights for the researchers to conduct analysis and to discuss findings and their implications.

2. Questionnaire Development and Data Collection Process

The questionnaire was developed after an analysis of the literature; as shown in Table 1, the questions were adopted and adapted from published sources. Also, the questionnaire was checked by five university professors and three practitioners, and their comments were applied. This ensured the content and face validity of the scale. Then, the questionnaire was tested in three stages, each with five respondents. At the end of each stage, the questions which needed revision were modified. At the end of the third stage, it was found that the questionnaire was clear, understandable and meaningful. The question items are presented in Table 2. After the pre-test stage, a purposive sampling method was used, and 240 questionnaires were distributed to the medical doctors working in 5 hospitals who had at least three months of experience with CPOE systems. After three months, 166 questionnaires usable for analysis were collected, indicating a response rate of 68.3%.

III. Results

As seen in Table 3, the majority of respondents were female. Moreover, it was found that the majority of the respondents were between 31-40 and 20-30 years old. It was also found that the majority of respondents had more than 9 years experience of working with computers in general, and 45.2% had more than 1 year experience with the CPOE system. The results also indicated that a majority of the respondents had practiced medicine for more than 5 years. These facts imply that the respondents had sufficient IT knowledge, experience with the CPOE system, and medical experience to understand the questionnaire.

The scale was tested for multivariate normality and, as shown in Table 1, it was found that the assumption of normality is violated for the variables ease of use and error reduction. Due to the lack of normality, the partial least square (PLS) path modelling technique-as the second generation of the multivariate analyses-should be used to analyse the relationships between the variables [35]. Therefore, SmartPLS2.0.M3 was used to analyze the data.

The reliability of the scale was examined by Cronbach's α and composite reliability (CR). As shown in Table 1, they exceeded 0.7 for all the variables. This implies that the scale has excellent reliability. The validity of the scale was examined by confirmatory factor analysis. As shown in Table 2, all the question items had a loading range from 0.76 to 0.90 (p < 0.001). Moreover, it was observed that the Fornell and Larcker criterion had been met. It was also found that items loading on their corresponding factors were greater than their cross loadings. Also, the average variance extracted values for all the variables exceed 0.5. This confirms the convergent and discriminate validity of the scale.

As shown in Figure 1, the results illustrated that relationships between CPOE user interface consistency (β = 0.415, t = 7.4, p < 0.01), and error prevention (β = 0.300, t = 5.5, p < 0.01) with ease of use are positive and significant. Ease of use (β = 0.293, t = 4.4, p < 0.01) and information quality (β = 0.408, t = 5.4, p < 0.01) of CPOE were also found to have positive significant influences on prescribing error reduction. These results are summarized in Table 4.

As shown in Table 5, 56% of the respondents believed that the system had reduced prescribing errors related to drug allergy. It was also found that 54.2% believed that using the system had reduced the likelihood of prescribing medicines which could have caused drug interaction. Also, 56.7% of the doctors believed that using the CPOE system had reduced the likelihood of drug dosing errors in their prescriptions. The results also indicated that 50% of the respondents maintained that the system had improved patient safety.

IV. Discussion

This research empirically validated a model regarding the impacts of CPOE usability on the reduction of prescribing errors. The results validated and confirmed the suggestions of the scholars in this field. Zhang et al. [4] suggest that the clinical systems should be designed in such a way that they prevent errors in the first place. Whether the features of a CPOE can reduce prescribing errors needs to be verified. Our statistical analyses found that the ease of use of the CPOE reduces doctors' prescribing errors. This is consistent with the conclusions of Khajouei et al. [42], who stated that doctors using a CPOE had a positive impression of ease of use and its influences on their efficiency and medication safety. This is because the ease of use of the CPOE reduces doctors' mental and cognitive loads. Therefore, they can better concentrate on other aspects of their job, including diagnosis and prescribing processes. This leads to the reduction of their prescribing errors.

Moreover, past research has suggested that the ease of use of an information system is the result of some system features [40]. To validate this relationship, this paper empirically examined the influence of interface consistency and system error prevention on the ease of use. The results demonstrated that the consistency of the user interface across various screens and system error prevention features result in the ease of use of the system. Researchers have suggested that CPOE interface consistency [4,10,43] and error prevention [19] are important factors in the design of clinical systems. The results provide empirical support for the (indirect) effects of user interface consistency and error prevention via the ease of use of the system on the reduction of prescribing errors. The results imply that in order for healthcare managers to reduce prescribing errors in hospitals, improve patient safety, and deliver quality care services, they need to improve the user interface features of CPOE systems, such as ease of use, error prevention, and consistency.

Furthermore, this study found that information quality is the most significant antecedent of prescribing error reduction. This suggests that information specifications should meet the doctors' requirements. Quality information should be accurate, relevant, and timely. The results imply that producing quality information is important in CPOE systems, which have been designed to improve the care output by obtaining and sharing patients' information and making decisions based upon it. Decision making in healthcare is information intensive; therefore, the quality of information is an important factor in care. Indeed, effective care is dependent on acquiring and using high quality information [44-46]. Having access to quality information, doctors can efficiently prescribe suitable and effective medicine and medications for the patient. When the system provides quality information for doctors regarding the medical and health history of patients, as well as the effects of drugs on patients, it helps them avoid prescribing errors. For instance, if a doctor prescribes a dexamethasone ampoule for a patient who already has diabetes, the new prescribed medicine can have negative effects on the patient. However, by having access to quality information through CPOE regarding the past health history of the patient, the doctor will avoid prescribing such drugs for patients. Moreover, if a doctor prescribes certain medicines for a patient, the system can check the prescription content and warn the doctor by providing good quality information regarding any drug interactions or drug dosing errors existing in the prescription. This reduces the likelihood of prescribing errors and improves patient safety practices in hospitals. On the other hand, a CPOE system producing low quality information threatens patients' safety due to doctors' reliance on erroneous information of the system. This suggests that if the CPOE system does not provide good quality information to doctors, this may increase their prescribing errors and threatens patient safety. Therefore, to reduce prescribing errors, a CPOE system should be designed in such a way that it produces accurate, relevant, and timely information for doctors.

From a methodological point of view, this research applied advanced quantitative analysis techniques to ensure the reliability and validity of the scale as well as examining the hypotheses. This study is the first research that developed a reliable and valid scale to measure prescribing error reduction. The scale developed in this research can be used in future studies to empirically measure the impact of systems on the reduction of prescribing errors. Moreover, since this research collected data from a methodologically sufficient sample size, its findings are generalizable compared to the prior quantitative research, which had used small sample sizes.

Overall, the results suggest that CIS designers and developers need to incorporate user interface consistency, error prevention, ease of use, and quality information in the system to reduce the likelihood of prescribing errors occurring and, ultimately, improve patient safety and medical service quality. This is consistent with the findings of Kekre et al. [43] who found that the usability of a system plays an important role in its success or failure. They found that "usability is the most important factor to end users" [39]. Therefore, system designers and developers should focus on the improvement of system usability to ensure the success of the system [43], such as reduction of errors.

The authors acknowledge several limitations of this study. First, the reduction of medical errors by a system is not only influenced by the usability features incorporated in the research model of this paper; other system features might have a significant impact on the reduction of medical errors. Therefore, future studies need to consider the impacts of other system features on the reduction of prescribing errors. Furthermore, future studies need to examine more complex models and explore the inter-relations between the system features and their direct and indirect impact on the reduction of prescribing errors. Another limitation is that this study used self-report data, which limits the strength of the findings. Future attempts should collect and use objective data to test and validate the model. Moreover, future research should collect data from a variety of sources to cross validate the collected data and findings. This research examined the impact of system usability features on the reduction of prescribing errors, but it did not address the impact of such systems on other care outcomes, such as medical staff workload, decision making quality, communication, facilitation of care, etc. Another potential opportunity for research is that this paper tested the impacts of CPOE usability on the reduction of medical errors. Future research can use other CIS modules, such electronic health record systems, to validate the model. Also, this research did not measure the influence of doctors' IT knowledge, experience/familiarity with CPOE systems, and medical experience on the effectiveness of their use of CPOE systems in relation to prescribing errors.

Figures and Tables

Figure 1

The results of the hypotheses. ^**p < 0.01.

Table 1

Scale, source, and reliability

SD: standard deviation, CR: composite reliability, AVE: average variance extracted.

Table 2

Item loadings

VIF: variance inflation factor.

Table 3

Profile of the respondents (n = 166)

Table 4

Results of the hypotheses

Table 5

Impacts of CPOE on the reduction of prescribing errors

CPOE: computerized physician order entry.

Notes

No potential conflict of interest relevant to this article was reported.

References

1. Blaser R, Schnabel M, Biber C, Baumlein M, Heger O, Beyer M, et al. Improving pathway compliance and clinician performance by using information technology. Int J Med Inform. 2007; 76(2-3):151–156.

2. Bates DW, Cohen M, Leape LL, Overhage JM, Shabot MM, Sheridan T. Reducing the frequency of errors in medicine using information technology. J Am Med Inform Assoc. 2001; 8(4):299–308.

3. Bosman RJ. Impact of computerized information systems on workload in operating room and intensive care unit. Best Pract Res Clin Anaesthesiol. 2009; 23(1):15–26.

4. Zhang J, Johnson TR, Patel VL, Paige DL, Kubose T. Using usability heuristics to evaluate patient safety of medical devices. J Biomed Inform. 2003; 36(1-2):23–30.

5. Blendon RJ, DesRoches CM, Brodie M, Benson JM, Rosen AB, Schneider E, et al. Views of practicing physicians and the public on medical errors. N Engl J Med. 2002; 347(24):1933–1940.

6. Effken JA, Loeb RG, Kang Y, Lin ZC. Clinical information displays to improve ICU outcomes. Int J Med Inform. 2008; 77(11):765–777.

7. Lee DH, Lee SM, Schniederjans MJ. Medical error reduction: the effect of employee satisfaction with organizational support. Serv Ind J. 2001; 31(8):1311–1325.

8. Holden RJ. Physicians' beliefs about using EMR and CPOE: in pursuit of a contextualized understanding of health IT use behavior. Int J Med Inform. 2010; 79(2):71–80.

9. Eslami S, de Keizer NF, Abu-Hanna A. The impact of computerized physician medication order entry in hospitalized patients; a systematic review. Int J Med Inform. 2008; 77(6):365–376.

10. Oztekin A, Nikov A, Zaim S. UWIS: an assessment methodology for usability of Web-based information systems. J Syst Softw. 2009; 82(12):2038–2050.

11. Han YY, Carcillo JA, Venkataraman ST, Clark RS, Watson RS, Nguyen TC, et al. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics. 2005; 116(6):1506–1512.

12. Maenpaa T, Suominen T, Asikainen P, Maass M, Rostila I. The outcomes of regional healthcare information systems in health care: a review of the research literature. Int J Med Inform. 2009; 78(11):757–771.

13. Venkatesh V, Zhang X, Sykes TA. "Doctors do too little technology": a longitudinal field study of an electronic healthcare system implementation. Inf Syst Res. 2011; 22(3):523–546.

14. Reckmann MH, Westbrook JI, Koh Y, Lo C, Day RO. Does computerized provider order entry reduce prescribing errors for hospital inpatients? A systematic review. J Am Med Inform Assoc. 2009; 16(5):613–623.

15. Nahm ES, Vaydia V, Ho D, Scharf B, Seagull J. Outcomes assessment of clinical information system implementation: a practical guide. Nurs Outlook. 2007; 55(6):282–288.

16. Oztekin A, Kong ZJ, Uysal O. UseLearn: a novel checklist and usability evaluation method for eLearning systems by criticality metric analysis. Int J Ind Ergon. 2010; 40(4):455–469.

17. Weir CR, Staggers N, Laukert T. Reviewing the impact of computerized provider order entry on clinical outcomes: the quality of systematic reviews. Int J Med Inform. 2012; 81(4):219–231.

18. Amarasingham R, Plantinga L, Diener-West M, Gaskin DJ, Powe NR. Clinical information technologies and inpatient outcomes: a multiple hospital study. Arch Intern Med. 2009; 169(2):108–114.

19. Corrao NJ, Robinson AG, Swiernik MA, Naeim A. Importance of testing for usability when selecting and implementing an electronic health or medical record system. J Oncol Pract. 2010; 6(3):120–124.

20. Su KW, Liu CL. A mobile Nursing Information System based on human-computer interaction design for improving quality of nursing. J Med Syst. 2012; 36(3):1139–1153.

21. Babaei M, Ganjalikhani M. Medical informatics: on an ongoing road to final destination. Sri Lanka J Biomed Inform. 2010; 1(3):186–190.

22. Gonzalez M, Masip L, Granollers A, Oliva M. Quantitative analysis in a heuristic evaluation experiment. Adv Eng Softw. 2009; 40(12):1271–1278.

23. Hair JF, Black WC, Babin BJ, Anderson RE. Multivariate data analysis. 7th ed. Upper Saddle River (NJ): Prentice-Hall;2010.

24. Joshi A, Sarda NL, Tripathi S. Measuring effectiveness of HCI integration in software development processes. J Syst Softw. 2010; 83(11):2045–2058.

25. Cooper RB. Decision production: a step toward a theory of managerial information requirements. In : Proceedings of the Fourth International Conference on Information Systems; 1983 Dec 15-17; Houston, TX. p. 251–267.

26. Pai FY, Huang KI. Applying the Technology Acceptance Model to the introduction of healthcare information systems. Technol Forecast Soc Change. 2011; 78(4):650–660.

27. Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 1989; 13(3):319–340.

28. Chen SY, Macredie RD. The assessment of usability of electronic shopping: a heuristic evaluation. Int J Inf Manage. 2005; 25(6):516–532.

29. Despont-Gros C, Mueller H, Lovis C. Evaluating user interactions with clinical information systems: a model based on human-computer interaction models. J Biomed Inform. 2005; 38(3):244–255.

30. Thong JY, Hong W, Tam KY. Understanding user acceptance of digital libraries: what are the roles of interface characteristics, organizational context, and individual differences? Int J Hum Comput Stud. 2002; 57(3):215–242.

31. Ammenwerth E, Rauchegger F, Ehlers F, Hirsch B, Schaubmayr C. Effect of a nursing information system on the quality of information processing in nursing: an evaluation study using the HIS-monitor instrument. Int J Med Inform. 2011; 80(1):25–38.

32. Devaraj S, Kohli R. Performance impacts of information technology: is actual usage the missing link? Manage Sci. 2003; 49(3):273–289.

33. Conway JM, Lance CE. What reviewers should expect from authors regarding common method bias in organizational research. J Bus Psychol. 2010; 25(3):325–334.

34. Fleenor JW, Smither JW, Atwater LE, Braddy PW, Sturm RE. Self-other rating agreement in leadership: a review. Leadersh Q. 2010; 21(6):1005–1034.

35. Hair JF, Ringle CM, Sarstedt M. PLS-SEM: indeed a silver bullet. J Mark Theory Pract. 2011; 19(2):139–152.

36. Ahn T, Ryu S, Han I. The impact of Web quality and playfulness on user acceptance of online retailing. Inf Manag. 2007; 44(3):263–275.

37. Goetzinger L, Park J, Lee YJ, Widdows R. Value-driven consumer e-health information search behavior. Int J Pharm Healthc Mark. 2007; 1(2):128–142.

38. Kuan HH, Bock GW, Vathanophas V. Comparing the effects of Website quality on customer initial purchase and continued purchase at e-commerce Websites. Behav Inf Technol. 2008; 27(1):3–16.

39. Venkatesh V, Davis FD. A theoretical extension of the technology acceptance model: four longitudinal field studies. Manage Sci. 2000; 46(2):186–204.

40. Smith RJ, Eroglu C. Assessing consumer attitudes toward off-site customer service contact methods. Int J Logist Manag. 2009; 20(2):261–267.

41. Mwachofi A, Walston SL, Al-Omar BA. Factors affecting nurses' perceptions of patient safety. Int J Health Care Qual Assur. 2011; 24(4):274–283.

42. Khajouei R, Wierenga PC, Hasman A, Jaspers MW. Clinicians satisfaction with CPOE ease of use and effect on clinicians' workflow, efficiency and medication safety. Int J Med Inform. 2011; 80(5):297–309.

43. Kekre S, Krishnan MS, Srinivasan K. Drivers of customer satisfaction for software products: implications for design and service support. Manage Sci. 1995; 41(9):1456–1470.

44. Lammintakanen J, Saranto K, Kivinen T. Use of electronic information systems in nursing management. Int J Med Inform. 2010; 79(5):324–331.

45. Kim J, Kim S, Jung Y, Kim EK. Status and problems of adverse event reporting systems in Korean hospitals. Healthc Inform Res. 2010; 16(3):166–176.

46. Hwang JI, Lee SI, Park HA. Barriers to the operation of patient safety incident reporting systems in Korean general hospitals. Healthc Inform Res. 2012; 18(4):279–286.

TOOLS

Similar articles