Safety and Usability Guidelines of Clinical Information Systems Integrating Clinical Workflow: A Systematic Review

Yura Lee; Min-Young Jung; Gee Won Shin; Sangwoo Bahn; Taezoon Park; Insook Cho; Jae-Ho Lee

doi:10.4258/hir.2018.24.3.157

Journal List > Healthc Inform Res > v.24(3) > 1099769

Go to TopGo to Top Go to BottomGo to Bottom

TOOLS

Lee, Jung, Shin, Bahn, Park, Cho, and Lee: Safety and Usability Guidelines of Clinical Information Systems Integrating Clinical Workflow: A Systematic Review

Original Article

Healthcare Informatics Research 2018; 24(3): 157-169.

Published online: 31 July 2018

DOI: https://doi.org/10.4258/hir.2018.24.3.157

Safety and Usability Guidelines of Clinical Information Systems Integrating Clinical Workflow: A Systematic Review

Yura Lee, MD, PhD¹, Min-Young Jung, RN, MS¹, Gee Won Shin, MS², Sangwoo Bahn, PhD³, Taezoon Park, PhD⁴, Insook Cho, RN, PhD⁵, Jae-Ho Lee, MD, PhD^1,⁶

¹Department of Biomedical Informatics, Asan Medical Center, Seoul, Korea.

²Department of Industrial Engineering, Human Interface System Lab, Seoul National University, Seoul, Korea.

³Department of Industrial and Management Systems Engineering, Kyung Hee University, Seoul, Korea.

⁴Department of Industrial and Information Systems Engineering, Soongsil University, Seoul, Korea.

⁵Department of Nursing, Inha University, Incheon, Korea.

⁶Department of Emergency Medicine, University of Ulsan Collage of Medicine, Seoul, Korea.

Corresponding Author: Jae-Ho Lee, MD, PhD. Department of Emergency Medicine, University of Ulsan College of Medicine, 88 Olympic-ro 43-gil, Songpa-gu, Seoul 05505, Korea. Tel: +82-2-3010-3350, jaeholee@amc.seoul.kr

Received 3 May 2018 Revised 8 July 2018 Accepted 12 July 2018

The Korean Society of Medical Informatics

(open-access):

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Objectives

The usability of clinical information systems (CISs) is known to be an essential consideration in ensuring patient safety as well as integrating clinical flow. This study aimed to determine how usability and safety guidelines of CIS consider clinical workflow through a systematic review in terms of the target systems, methodology, and guideline components of relevant articles.

Methods

A literature search was conducted for articles published from 2000 to 2015 in PubMed, Cochrane, EMBASE, Web of Science, and CINAHL. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement method was employed. Articles containing recommendations, principles, and evaluation items for CIS usability and safety were included. The selected articles were classified according to article type, methodology, and target systems. Taking clinical workflow into consideration, the components of guidelines were extracted and classified.

Results

A total of 7,401 articles were identified by keyword search. From the 76 articles remaining after abstract screening, 15 were selected through full-text review. Literature review (n = 7) was the most common methodology, followed by expert opinions (n = 6). Computerized physician order entry (n = 6) was the most frequent system. Four articles considered the entire process of clinical tasks, and two articles considered the principles of the entire process of user interface affecting clinical workflow. Only two articles performed heuristic evaluations of CISs.

Conclusions

The usability and safety guidelines of CISs need improvement in guideline development methodology and with consideration of clinical workflow.

Keywords: Hospital Information Systems, Patient Safety, User-Computer Interface, Guideline, Workflow

I. Introduction

Many errors regarding clinical information systems (CISs) have been reported [1 2 3 4]. Research on errors and potential hazards has led to review and updating of CISs to promote patient safety [5 6 7]. Efforts to reduce errors have assessed CISs in light of human factors [8]. As a result, guidelines have been developed and distributed to consider usability from the CIS design stage.

However, efforts to pursue patient safety have sometimes had conflicting results regarding usability. An alert function that can filter out simple input errors may interfere with the flow of work, which may adversely affect usability [9 10 11]. On the other hand, functions to improve usability can cause errors. For a novice user, shortcuts or abbreviations for work efficiency could threaten patient safety. Therefore, user-centered CIS design guidelines are needed, not only to prevent errors but also to improve clinical workflow [12 13 14 15].

The accumulation of structured (coded) data enables the development of decision support rules. Improvements in network speed allow immediate feedback to users, with processing of evidence-based medicine and patient data. Consequently, system-level controls to reduce user mistakes (or for best practices) have become easier. Moreover, professional considerations of usability are needed as CISs become more complex and sophisticated because novice personnel regularly enter the field and must be adapted to the system [16 17 18]. We should consider the diverse functions of CISs and various fields of labor in CIS design and maintenance, as well as upgrades in CIS guidelines.

We conducted this study to help establish guidelines to improve the safety and usability of CIS through a systematic review and to analyze how the clinical workflow and usability principles are reflected differently in existing articles [12 19 20]. We divided the steps of clinical workflow and performed a guideline review. Furthermore, this review sought clues to balancing CIS usability and safety.

II. Methods

This review of CIS guidelines was based on a systematic review of the literature based on the results of searches on the related keywords. Detailed recommendations and principles of evaluation were extracted by reviewing the selected articles, whose components were classified and reassembled according to various criteria referenced in the selected literature to analyze how they affect clinical workflow. The present study protocol was reviewed and approved by the Institutional Review Board of Asan Medical Center (No. 2016-0980).

1. Systematic Review of Relevant Articles

Literature searches were performed in PubMed, Cochrane, EMBASE, Web of Science, and CINAHL using keywords related to health information technology (HIT), safety, and usability in guideline development published in English between January 2000 and December 2015 (Table 1). The protocol for this review was guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement.

The keywords used in the search were selected through discussions by the research team. Keywords were classified into three categories: guideline, HIT, and safety/usability. Each category was searched using OR for keywords and synonyms and AND between categories.

Articles containing recommendations, principles, and evaluation items for CIS design, usage, or evaluation were included. On the other hand, those with ‘content vague or with too narrow focus’, ‘issues irrelevant to safety or usability principle’, ‘no suggestion for guideline/recommendation’, or ‘unable to find full-text article’ were excluded.

Two researchers independently checked the titles and abstracts of the searched articles and conducted the first selection (Figure 1). Upon agreement between these two researchers, the selection of the documents to be included in the final evaluation was made through discussion with a third researcher. After securing the full text for the final selection, secondary review was performed independently by the first two researchers. Conflicts were resolved, and the final selection for the full literature review was made through a discussion by all three researchers. In case of non-agreement, the research team reached a decision through consultation.

2. Review of Selected Articles and Extraction of the Guideline Components

The selected articles were reviewed in terms of objectives, the type of articles, and nationality. The target system and methodology of the articles were also classified according to whether they presented a guideline or principle based on existing knowledge (e.g., literature review), empirical content (e.g., expert opinion or group discussion), user opinion (e.g., user survey or testing), or heuristic evaluations. The guidelines, recommendations, principles, or evaluation items from the articles were then extracted as guideline components.

3. Review and Re-classification of Guideline Components Affecting Clinical Workflow

We stratified the clinical workflow according to the following types of actions. A user identifies and selects patients’ data, records medical documents, orders medications or investigations, gets feedback from the system by alert or Clinical Decision Support (CDS), and manages and maintains the system [12 21 22 23 24]. Therefore, the guideline components were classified according to clinical tasks (data identification & selection, document entry, order entry, clinical decision support & alert, and management) and usability principles (screen recognition, data view & entry, running & control, and feedback) following considerations of clinical workflow. The research team determined the final categories and methods for classifying the guideline components through discussion.

III. Results

Among a total of 7,401 searched articles, 15 articles were finally selected through the systematic review [7 9 10 17 25 26 27 28 29 30 31 32 33 34 35]. Table 2 summarizes the characteristics of the studies included in this systematic review. All but one of the studies were written in the United States (73%, 11/15) or Canada (20%, 3/15); the other was from the UK (Table 2). Almost half (47%, 7/15) of the articles were original research papers, followed by perspective, opinion, and commentary (5/15) and review articles (3/15). The article types were assigned based on information from the journals. For the target system, Computerized Physician Order Entry (CPOE) was the most frequent system, whereas Electronic Medical Record (EMR) was the least frequent (Table 3). Article review was the most common methodology (7/15), followed by expert opinion (6/15). An article review was followed by additional analysis in the majority of cases (4/7). Three articles concerned the results of user testing or surveys, and only two articles conducted heuristic evaluations.

Clinical components such as ‘drug-patient age checking’ were classified under ‘clinical tasks’, and usability components such as ‘clearly legible font’ were classified under ‘usability principles’ [10 34]. There were some conflicts within a given category. For example, default values for medication prescription are sometimes recommended (“The system should provide for selection from the dosages and forms that are available and appropriate for a given medication,” by Bell et al. [26]) and sometimes not (“System should limit or not use defaults for medications,” by Carvalho et al. [28]). A brief explanation, such as ‘Only enter default values for drug, dose, frequency, and route if it will always be correct’, balanced safety and usability [7].

Table 4 shows the results of the categorization of guideline components according to clinical workflow. Four of the 15 selected articles [10 17 26 32] were classified as applicable to the entire process of the clinical tasks affecting clinical workflow. For example, “Content should be limited to 1–2 lines, with a justification separated by white space” by Horsky et al. [10] for data identification & selection; “Monitor use of C/P functions in record preparation and limit use of ‘boilerplate’ content across records” by Zahabi et al. [17] for document entry; and “Conducts a verification process to ensure that all medications comply with recommended dosing based on current evidence-based literature” by the Institute for Safe Medication Practices (ISMP) [32] for order entry and CDS & alert. Within the clinical tasks, CDS and alert (13/15) and system management (9/12) were commented on by a majority of the articles. General principles of alert systems, such as alert priority or reduction of alert fatigue, were most frequently discussed in the articles (Table 5).

Two articles [17 32] were classified as applicable to the entire process of selecting the usability principles affecting clinical workflow (Table 5). Running & control was the most frequently considered principle (10/15), and data view & entry was the least (4/15). The article by Zahabi et al. [17] concerned the clinical tasks and usability principles affecting clinical workflow. The methodologies of the 5 commented articles were article review [10 17], group discussion [26 32], and expert opinion [31].

IV. Discussion

In this study, we selected articles presenting guidelines or principles for CIS safety and usability that should be taken into account in the design, use, and management of such systems. We reviewed the target system and methodology and reviewed and summarized the guidelines and evaluation items to date. Also, we divided the steps of clinical workflow into the aspects of clinical tasks and usability principles. Articles focused on specific tasks (CDS and alert) (13/15). Also, among the usability principles, the components of data view & entry were relatively small (4/15). Articles showed a lack of diversity in clinical task and principle for each step of clinical workflow. All articles with inclusive clinical workflow components were based on existing knowledge or experience [10 17 26 31]. To our knowledge, no study has reviewed the CIS safety and usability guidelines according to the detailed steps of clinical workflow.

1. Balance in Conflicts of Safety and Usability of CIS

Although improvement of usability is directed toward pursuing patient safety, some cases had conflicts, as mentioned above regarding default values for medication prescription. Recommendations like “Omit items for which the information is not available to the user,” which are intended to reduce cognitive load, could threaten patient safety if clinical workflow is not taken into consideration. Similarly, “Cluster related information on the same screen” [17] and “Avoid too much information on the screen at one time” [29] seem to be mutually contradictory without a consideration of the clinical situation.

Participation of clinicians and feedback to users can mitigate the conflicts in safety versus usability, as comments on the role of clinicians in system adaptation and order set pointed out [32 35]. Some override functions have the risk of bypassing critical alerts, though they prevent alert fatigue, which reduces workflow efficiency [11 12 13 14 26 27 28]. However, profiling and reporting the history of alert overrides can be helpful in cases where critical alerts are of overridden [26 27]. Therefore, the consideration of clinical workflow from the user's point of view is important to balance safety and usability.

2. Challenges of Previous Studies: Localized Methodologies and Target Systems

As the terms of the systematic review were set to English articles, and all the selected articles were North American and UK studies. Among the selected articles were seven original research articles according to the publishers' classifications, but expert survey/group discussions constituted the majority of these articles. Few methodologies of heuristic evaluation were given [2], and only one article conducted a retrospective cohort study [35]. The majority included existing knowledge (article review, expert opinion, or group discussion), and there were few articles on user testing/surveys.

Much of the content concerned CDS, actions, and management, but that on data presentation was relatively small. Only one-third (5/15) of the articles included all the steps of clinical tasks and/or usability principles affecting clinical workflow. Moreover, these 5 articles were based on existing knowledge, not up-to-date user experiences or experimental evidence. Article reviews and empirical knowledge are essential for the improvement of past error and usability problems. However, considering improvements in the speed of system performance and the development of infrastructure such as a CIS network, more active experiments and studies of systems-in-use and/or systems-in-advance will be needed.

There was no specific recommendation or consideration of the size or specificity of the medical institutions except for one document related to a GP system in primary care, Avery et al. [27] (Table 2). The use of computers in medical institutions of various sizes has expanded the management of patient data. The need for the development of principles of information systems to ensure the efficient and safe exchange of information with primary care institutions and hospitals has increased.

Our findings were similar to those of previous systematic reviews on CIS safety/usability, in that most of the articles lacked a consideration of overall clinical workflow. In methodology, the majority were based on existing knowledge or empirical content and lacked an explicit theoretical framework or model, as Yen and Bakken [19] commented.

3. Limitations

In this study, the components of more specific functions or tasks (e.g., patient lists, dashboards, or override alerts) were not included. Also, non-English articles and guidelines were not included, and as this study was a part of the guideline development process, experimental articles not containing guidelines or principles were excluded. This factor might have served to exclude more experimental research articles.

4. Conclusion

There was a lack of consideration of the entire clinical workflow in the selected articles. Also, in many cases, guidelines were developed through the synthesis of existing knowledge rather than through user testing or heuristic evaluations. Development of CIS guidelines affecting clinical workflow is needed for usability and patient safety. To promote the safety and usability of CIS, more user-oriented guidelines that take into account the clinical work-flow are needed.

Figures and Tables

Figure 1

Flowchart of the systematic review.

Table 1

Selected keywords and their categories for the literature search

Each category was searched by OR for keywords and synonyms, and by AND between categories.

Table 2

Summary of the selected articles

CDS: Clinical Decision Support, CPOE: Computerized Physician Order Entry, EMR: Electronic Medical Record, EHR: Electronic Health Record, ISMP: Institute for Safe Medication Practices, VA CPRS: US Department of Veterans Affairs Computerized Patient Record System.

Table 3

Target systems and methodologies of selected articles

CPOE: Computerized Physician Order Entry, CDS: Clinical Decision Support, EMR: Electronic Medical Record, CIS: Clinical Information Systems, ISMP: Institute for Safe Medication Practices.

Table 4

Components of guideline/principles for clinical tasks or usability principles affecting clinical workflow

CDS: Clinical Decision Support, ISMP: Institute for Safe Medication Practices.

Table 5

Guideline components for clinical tasks and usability principles considering workflow in selected articles

EMR: Electronic Medical Record, CDS: Clinical Decision Support.

Acknowledgments

This study was supported by a grant of the Korean Health Technology R&D Project, Ministry of Health & Welfare, Korea (No. HI15C1101).

Notes

^{Conflict of Interest} No potential conflict of interest relevant to this article was reported.

References

1. Menachemi N, Collum TH. Benefits and drawbacks of electronic health record systems. Risk Manag Healthc Policy. 2011; 4:47–55.

2. Friedberg MW, Chen PG, Van Busum KR, Aunon F, Pham C, Caloyeras J, et al. Factors affecting physician professional satisfaction and their implications for patient care, health systems, and health policy. Rand Health Q. 2014; 3(4):1.

3. Campbell EM, Sittig DF, Ash JS, Guappone KP, Dykstra RH. Types of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc. 2006; 13(5):547–556.

4. Harrison MI, Koppel R, Bar-Lev S. Unintended consequences of information technologies in health care: an interactive sociotechnical analysis. J Am Med Inform Assoc. 2007; 14(5):542–549.

5. Blumenthal D, Tavenner M. The “meaningful use” regulation for electronic health records. N Engl J Med. 2010; 363(6):501–504.

6. Sengstack P. CPOE configuration to reduce medication errors. J Healthc Inf Manag. 2010; 24(4):26–34.

7. Hyman D, Laire M, Redmond D, Kaplan DW. The use of patient pictures and verification screens to reduce computerized provider order entry errors. Pediatrics. 2012; 130(1):e211–e219.

8. Brown CL, Mulcaster HL, Triffitt KL, Sittig DF, Ash JS, Reygate K, et al. A systematic review of the types and causes of prescribing errors generated from using computerized provider order entry systems in primary and secondary care. J Am Med Inform Assoc. 2017; 24(2):432–440.

9. Bates DW, Kuperman GJ, Wang S, Gandhi T, Kittler A, Volk L, et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc. 2003; 10(6):523–530.

10. Horsky J, Schiff GD, Johnston D, Mercincavage L, Bell D, Middleton B. Interface design principles for usable decision support: a targeted review of best practices for clinical prescribing interventions. J Biomed Inform. 2012; 45(6):1202–1216.

11. Prgomet M, Li L, Niazkhani Z, Georgiou A, Westbrook JI. Impact of commercial computerized provider order entry (CPOE) and clinical decision support systems (CDSSs) on medication errors, length of stay, and mortality in intensive care units: a systematic review and meta-analysis. J Am Med Inform Assoc. 2017; 24(2):413–422.

12. Eisenberg F, Barbell AS. Computerized physician order entry: eight steps to optimize physician workflow. J Healthc Inf Manag. 2002; 16(1):16–18.

13. Tu SW, Musen MA, Shankar R, Campbell J, Hrabak K, McClay J, et al. Modeling guidelines for integration into clinical workflow. Stud Health Technol Inform. 2004; 107(Pt 1):174–178.

14. Niazkhani Z, Pirnejad H, Berg M, Aarts J. The impact of computerized provider order entry systems on inpatient clinical workflow: a literature review. J Am Med Inform Assoc. 2009; 16(4):539–549.

15. Khajouei R, Jaspers MW. The impact of CPOE medication systems' design aspects on usability, workflow and medication orders: a systematic review. Methods Inf Med. 2010; 49(1):3–19.

16. Ellsworth MA, Dziadzko M, O'Horo JC, Farrell AM, Zhang J, Herasevich V. An appraisal of published usability evaluations of electronic health records via systematic review. J Am Med Inform Assoc. 2017; 24(1):218–226.

17. Zahabi M, Kaber DB, Swangnetr M. Usability and safety in electronic medical records interface design: a review of recent literature and guideline formulation. Hum Factors. 2015; 57(5):805–834.

18. Middleton B, Bloomrosen M, Dente MA, Hashmat B, Koppel R, Overhage JM, et al. Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA. J Am Med Inform Assoc. 2013; 20(e1):e2–e8.

19. Yen PY, Bakken S. Review of health information technology usability study methodologies. J Am Med Inform Assoc. 2012; 19(3):413–422.

20. Zhang J, Johnson TR, Patel VL, Paige DL, Kubose T. Using usability heuristics to evaluate patient safety of medical devices. J Biomed Inform. 2003; 36(1-2):23–30.

21. Lauesen S, Younessi H. Six styles for usability requirements. In : Proceedings of the 4th International Workshop on Requirements Engineering: Foundation for Software Quality (REFSQ); 1998 Jun 8-9; Pisa, Italy. p. 155–166.

22. Dix A. Human-computer interaction. In : Liu L, Ozsu MT, editors. Encyclopedia of database systems. Boston (MA): Springer;2009. p. 1327–1331.

23. Nielsen J. Heuristic evaluation. In : Nielsen J, Marck RL, editors. Usability inspection methods. New York (NY): John Wiley & Sons;1994. p. 25–62.

24. Nielsen J. Usability engineering. San Francisco (CA): Morgan Kaufmann Publishers;1993.

25. Nolan TW. System changes to improve patient safety. BMJ. 2000; 320(7237):771–773.

26. Bell DS, Marken RS, Meili RC, Wang CJ, Rosen M, Brook RH, et al. Recommendations for comparing electronic prescribing systems: results of an expert consensus process. Health Aff (Millwood). 2004; Suppl Web Exclusives. W4-305-17.

27. Avery AJ, Savelyich BS, Sheikh A, Cantrill J, Morris CJ, Fernando B, et al. Identifying and establishing consensus on the most important safety features of GP computer systems: e-Delphi study. Inform Prim Care. 2005; 13(1):3–12.

28. Carvalho CJ, Borycki EM, Kushniruk A. Ensuring the safety of health information systems: using heuristics for patient safety. Healthc Q. 2009; 12(Spec No Patient):49–54.

29. Corrao NJ, Robinson AG, Swiernik MA, Naeim A. Importance of testing for usability when selecting and implementing an electronic health or medical record system. J Oncol Pract. 2010; 6(3):120–124.

30. Chan J, Shojania KG, Easty AC, Etchells EE. Usability evaluation of order sets in a computerised provider order entry system. BMJ Qual Saf. 2011; 20(11):932–940.

31. Zopf-Herling KM. Enhancing usability in EMR screen design. Comput Inform Nurs. 2011; 29(12):679–691.

32. Institute for Safe Medication Practices. Guidelines for standard order sets [Internet]. Horsham (PA): Institute for Safe Medication Practices;2010. cited at 2018 May 4. Available from https://www.ismp.org/guidelines/standard-order-sets.

33. McGreevey JD 3rd. Order sets in electronic health records: principles of good practice. Chest. 2013; 143(1):228–235.

34. Vartian CV, Singh H, Russo E, Sittig DF. Development and field testing of a self-assessment guide for computer-based provider order entry. J Healthc Manag. 2014; 59(5):338–352.

35. Green LA, Nease D Jr, Klinkman MS. Clinical reminders designed and implemented using cognitive and organizational science principles decrease reminder fatigue. J Am Board Fam Med. 2015; 28(3):351–359.

TOOLS

Similar articles