Conformance Analysis of Clinical Pathway Using Electronic Health Record Data

Sooyoung Yoo; Minsu Cho; Seok Kim; Eunhye Kim; So Min Park; Kidong Kim; Hee Hwang; Minseok Song

doi:10.4258/hir.2015.21.3.161

Abstract

Objectives

The objective of this study was to confirm the conformance rate of the actual usage of the clinical pathway (CP) using Electronic Health Record (EHR) log data in a tertiary general university hospital to improve the CP by reflecting real-world care processes.

Methods

We analyzed the application and matching rates of clinicians' orders with predefined CP order sets based on data from 164 inpatients who received appendectomies out of all patients who were hospitalized from August 2013 to June 2014. We collected EHR log data on patient information, medication orders, operation performed, diagnosis, transfer, and CP order sets. The data were statistically analyzed.

Results

The average value of the actual application rate of the prescribed CP order ranged from 0.75 to 0.89. The application rate decreased when the order date was factored in along with the order code and type. Among CP pre-operation, intra-operation, post-operation, routine, and discharge orders, orders pertaining to operations had higher application rates than other types of orders. Routine orders and discharge orders had lower application rates.

Conclusions

This analysis of the application and matching rates of CP orders suggests that it is possible to improve these rates by updating the existing CP order sets for routine discharge orders to reflect data-driven evidence. This study shows that it is possible to improve the application and matching rates of the CP using EHR log data. However, further research should be performed to analyze the effects of these rates on care outcomes.

I. Introduction

The clinical pathway (CP) is aimed at delivering high-quality medical services through the application of standardized, optimized care processes that are based on scientific evidence for inpatients who are treated for a particular diagnosis [1 2]. For this purpose, the structure of the CP provides a multi-disciplinary treatment plan and step-by-step execution of clinical activities for a particular problem [3 4]. In addition, the CP supports the proper allocation of healthcare resources as care and treatment techniques gradually develop. The efficiency of the CP, such as its impact on the effective management of care for inpatients during periods of hospitalization, has been reported in previous studies [5 6 7]. For these reasons, many hospitals have developed and utilized the CP in their actual practice. For example, over 80% of hospitals in the United States currently use the CP [8]. However, it is challenging to analyze the care processes that provide medical services in healthcare organizations because these processes are multi-disciplinary, highly dynamic, and complex. In addition, a continuous investment of resources is needed to maintain and improve the CP [4 9 10].

Considerable effort has been made in healthcare facilities to develop a suitable, evidence-based CP. It is possible to organize the CP in chronological order based on the clinical activities that are logged into the Electronic Health Record (EHR) system and to analyze the CP using process and data mining techniques [2 6 11 12 13 14]. These techniques are useful for gaining knowledge from the log data (timestamp and activity/task) that are saved in the system [4]. These techniques can also reduce the existing work time required to develop the CP. Previous studies have analyzed event logs to determine patterns and have also researched analysis methodology. However, to our knowledge, no study to date has analyzed the existing CP to recommend improvements that would better suit the needs and processes of the real-world medical environment.

In this study, we analyzed the application and matching rates of CP orders that were collected from an EHR system in a tertiary general university hospital in South Korea. We aimed to provide insight into the possibility of improvements in CP order sets utilizing conformance analysis.

II. Methods

1. Study Site

This study was performed at a 1,340-bed tertiary general university hospital located in the Seoul metropolitan area of Korea. The hospital has a fully paperless, comprehensive EHR system and has operated a CP Task Force Team (TFT) since 2007 to continuously develop and maintain the CP. Through regular meetings, the TFT board studied and determined the overall content of the CP. During this process, the TFT board monitored operating indicators of the CP, such as the application rate (the ratio of the number of CPapplied patients to the number of CP-targeted patients) and the completion rate of the CP (the ratio of the number of CP-completed patients to the number of CP-applied patients), for the continuous use of the existing CP. In addition, the TFT board discussed issues related to developing a new CP and improving the existing CP to better suit the needs of real-world clinical environments.

2. Collection of Event Log Data Relevant to the CP

As of 2013, the study site has developed and utilized approximately 150 CPs targeting hospitalized patients. In this study, the authors primarily analyzed the appendectomy CP, which has been in use since 2009. Because the appendectomy CP has been continuously improved since its development, this study focused on analyzing the most updated version of the CP. Thus, the authors analyzed the appendectomy CP based on patients who were enrolled in the appendectomy CP (out of all hospitalized patients) between August 9, 2013 and June 3, 2014, with the objective of investigating the application rate, matching rate, and improvement plan of the CP.

As a result, data were extracted from 164 hospitalized patients (9,296 events) in the appendectomy CP, which was applied for a total of three consecutive days. To ensure the accuracy of the analysis, we used data that had a structured value in the pre-process of the extracted data and excluded order data that were categorized as 'diet'. In addition, we used only active orders and excluded those medication orders that were entered by anesthesiology because anesthesiology orders are not targeted toward making improvements in the CP.

Table 1 shows the extracted data, including patient information, applied CP, operation, diagnosis, department/hospital transfer, and CP master order sets.

This study was approved by the Institutional Review Board of Seoul National University Bundang Hospital (IRB No: X-1310/221-904).

3. CP Conformance Analysis

The authors extracted the log data that were stored on the EHR system while providing medical services to hospitalized patients and performed a pre-processing conversion of the log data to data to allow for standard process derivation and pattern analysis using the process mining technique. The authors then derived a process model from the event log data that underwent the aforementioned pre-processing using the discovery method, which is a basic process mining analysis method. A process was then derived based on various mining algorithms, such as comp mining, which considers the event logs causal dependency, and heuristic mining, which considers the event logs frequency.

After deriving the process based on the data, we performed an analysis of the application and matching rates of the existing CP in the hospitals. The equation is shown below. Based on the data, improvements were suggested after comparison and evaluation of the previous CP data with the analysis of the matching rate and then deletion of the processes or patterns that showed low frequencies or no importance. For our analysis that considered the characteristics of the CP order, we used the following two types of order classification: order classification (test, procedure, or medication) and order type (pre-operation, intra-operation, post-operation, routine order, or discharge).

CP order application rate = \frac{Number of matching order codes in the CP order set}{Number of order codes in the CP order set}

CP order matching rate = \frac{1}{2} (1 - \frac{M_{c}}{N_{c}}) + \frac{1}{2} (1 - \frac{R_{L}}{N_{L}})

M_C: Number of order codes included in the CP order set but not shown in the event log data

N_C: Number of order codes included in the CP order set

R_L: Number of order codes included in the event log data but not shown in the CP order set

N_L: Number of order codes included in the event log data

III. Results

Figure 1 tracks the application of the overall CP orders from the extracted data. Most of the patients were treated by an order included in the CP order master (predefined CP order sets). However, a fraction of the orders were not commonly applied to all patients, and other orders that did not yet exist in the order master were applied to most patients. The matching graphic indicates a possible improvement to the CP to increase the matching rate.

Table 2 shows our analysis of the actual application rate of prescribed CP orders. The average value of the application rate based on four types of criteria ranged from 0.75 to 0.89, and the application rate decreased when the order date (ordering days) was factored in along with the order code and type.

Table 3 shows the results of the detailed analyses of criterion 3 and criterion 4 from Table 2. The predefined CP order sets for appendectomy were composed of 20 orders on the first day, 14 orders on the second day, and 15 orders on the third day. The application rates to actual usage for the first, second, and third days were 0.87, 0.75, and 0.60, respectively. The application rate was the highest on the first day, after which it progressively decreased. In addition, in the analysis of the application rate considering order type and the day order, most of the operation-related orders showed higher application rates (0.80-0.95); furthermore, routine and discharge order application rates were low (≤0.3).

Table 4 shows the analysis of the average acting rates for added orders that were not included in the predefined CP order sets out of all orders issued to patients. The average acting rate for added orders was 0.03, and the average acting rate for procedure orders was higher at 0.07. Preliminary orders showed a slightly higher acting rate than added orders; medication orders were the only type of orders that occurred in this category. By analyzing the acting rates on the order code level, we confirmed two order codes that showed a high acting rate (approximately 0.9) in the procedure order (see Figure 2).

We then combined the analysis of the application rate of the original CP for appendectomy with the analysis of the acting rate of the added order to create new adjusted CP order sets, excluding 4 codes that had application rates below 0.5 and adding 2 codes with acting rates above 0.8. The adjusted CP order set's conformance rates (application rate: 0.91, matching rate: 0.87) were higher than the original CP order set's conformance rates (application rate: 0.89, matching rate: 0.84) (Table 5).

IV. Discussion

Many hospitals already utilize the CP, which has a positive impact on both patient safety and hospital management, generating great interest in new and additional developments in the CP [5 6 7 8]. The CP is typically utilized in large hospitals because CP development and utilization requires an investment of a large workforce and extensive resources. However, even at large hospitals, it is not easy to invest in this amount of infrastructure to administer the CP and develop new CPs [4 15]. Huang et al. [11] previously conducted an electronic analysis of this issue to discover patterns in the CP based on the actual log data that occur chronologically in a patient's care flow. Those authors concluded that future studies should address the optimization of CP analysis and mechanisms by which hospitals can increase the actual application rate of the CP. In addition, another study developed a Clinical Pathway Analyzer (CPA) prototype that allowed for the analysis and comparison of pathways among patients [14]. Bouarfa et al. [16] analyzed 26 laparoscopic cholecystectomy operation activity logs to determine the pathway components that automatically exit the workflow without the participation of professionals in the clinical environment. Those authors reported the identification of outliers and found that it was possible to use the elicited results to improve operation workflow. Van de Klundert et al. [12] researched the development of a polynomial algorithm; however, those results were limited because they could not analyze various complex patient processes, and their adjustment rate was found to be low. The authors concluded that continuous development is needed because it is an important area of the health service industry.

Previous studies have utilized clinical activity logs to derive the CP through an automated algorithm or process mining technology. However, in this study, we did not develop a new CP. Instead, we analyzed the application and matching rates of an existing CP and demonstrated the possibility of making data-driven CP improvements. We have suggested improvements in the CP conformance rate (i.e., CP order application rate and CP order matching rate) by reducing the number of unused orders and adding orders that showed high adding and acting rates after comparing the actual orders with the CP order set. This methodology is distinct from previous methods, and it can be used to support a work process for monitoring and improving existing CPs in hospitals.

One limitation of this study is that increases in the application and matching rates of the CP were not studied in terms of their effects on clinical outcomes and administrative efficiency. The adjusted CPs were limited because they were developed under the premise that increases in application and matching rates would lead to increases in actual clinical outcomes and hospital administrative effects. Clinical factors were not considered. Therefore, future studies are needed to further examine various diseases and analyze the clinical and administrative effects that result from changes in the application and matching rates of the CP. These additional studies would verify the appropriateness of the research methods used in the present study. In addition, integrated analyses of data from multiple organizations are needed to confirm the appropriate threshold values of adequate application and matching rates of the CP.