Effects and Satisfaction of Medical Device Safety Information Reporting System Using Electronic Medical Record

Hye Jung Jang; Young Deuk Choi; Nam Hyun Kim

doi:10.4258/hir.2017.23.2.94

Abstract

Objectives

This paper describes an evaluation study on the effectiveness of developing an in-hospital medical device safety information reporting system for managing safety information, including adverse incident data related to medical devices, following the enactment of the Medical Device Act in Korea.

Methods

Medical device safety information reports were analyzed for 190 cases that took place prior to the application of a medical device safety information reporting system and during a period when the reporting system was used. Also, questionnaires were used to measure the effectiveness of the medical device safety information reporting system. The analysis was based on the questionnaire responses of 15 reporters who submitted reports in both the pre- and post-reporting system periods.

Results

Sixty-two reports were submitted in paper form, but after the system was set up, this number more than doubled to 128 reports in electronic form. In terms of itemized reporting, a total of 45 items were reported. Before the system was used, 23 items had been reported, but this increased to 32 items after the system was put to use. All survey variables of satisfaction received a mean of over 3 points, while positive attitude, potential benefits, and positive benefits all exceeded 4 points, each receiving 4.20, 4.20, and 4.13, respectively. Among the variables, time-consuming and decision-making had the lowest mean values, each receiving 3.53. Satisfaction was found to be high for system quality and user satisfaction, but relatively low for time-consuming and decision-making.

Conclusions

We were able to verify that effective reporting and monitoring of adverse incidents and the safety of medical devices can be implemented through the establishment of an in-hospital medical device safety information reporting system that can enhance patient safety and medical device risk management.

I. Introduction

The Korean market for medical devices has grown to 4.59 billion dollars, increasing by 6.6% compared to 2011 [1]. The world medical device market amounted to about 309 billion dollars in 2012, and it was expected to grow at an average annual rate of 6.7% after 2013 [2]. The use of medical devices will increase accordingly, and the possibility of consequent adverse incidents occurring will rise as well [3].

Since the Korea Ministry of Food and Drug Safety began to directly manage medical devices in 1997, a basic management framework has been established through the efforts of the ministry and the medical device industry in the form of a pre-market product approval system. The strengthening of a post-market management system is very significant in that it aims to further develop the current system into a high-quality medical device safety management system [4].

With the addition of provisions in the Medical Device Act to bolster the post-market safety management of medical devices, the Korea Ministry of Food and Drug Safety changed direction from focusing on the management of medical device safety at the pre-market approval stage and strengthened the legal basis for the continuous monitoring of medical device safety during the entire period of use through the management of all parties involved in medical device manufacturing, distribution, and usage, including medical device vendors, repairers, medical institutions, and patients [5]. In 2005, the Korea Ministry of Food and Drug Safety enacted in regulations of medical device safety management that include adverse incident reporting and a safety information management system necessary for the prevention of harm to public health resulting from the use of medical devices [6].

After the 2005 enactment of the regulations on reporting adverse incidents and safety management of medical devices, in 2010, the Korea Ministry of Food and Drug Safety reporting system was introduced, and the compilation of adverse incident and safety information reporting increased to over 100 cases [7]. With the Medical Device Safety Information Monitoring Center, a pilot project was launched in 2011, and the full-scale compilation of adverse incident and safety information reporting was implemented. Some 2000 cases related to medical device safety, including adverse incident reports, were compiled in 2012, and the number increased to 4,130 cases in 2013 [7].

Reflecting similar policies implemented by various advanced nations regarding post-market safety information collection and follow-up management, the Medical Device Act requires all medical device handlers, including manufacturers and users, to report adverse incidents and safety data related to the use of medical devices in order to manage the adverse incidents caused by medical devices in the post-market stage. Based on this, the law imposes a voluntary recall responsibility on medical device manufacturers and importers, requiring them to take immediate corrective measures, including recalls, for any device found to be harmful in terms of safety and effectiveness or defective in quality. Manufacturers and importers are also required to report the results to the Korea Ministry of Food and Drug Safety and to keep records of such reports for a minimum of 2 years [8].

Although the Korea Ministry of Food and Drug Safety's Department of Medical Device Safety Management is currently carrying out a pilot project to create a medical device safety information monitoring center for managing medical device safety information including adverse incidents, a standardized in-hospital reporting system has not yet been created. Previous medical device safety information reports were provided on paper; there were fewer reports and a lot of errors and inaccuracy.

The purpose of this study was to gain insight into how an in-hospital medical device safety reporting system can be designed to make the reporting of adverse incidents and safety information related to medical devices more active and more convenient.

II. Methods

1. System Development

1) System development environment

The reporting system consists of web servers, DB servers, management servers, user authentication servers, and a firewall. An in-hospital medical device safety information management system allows users to log in and submit in-hospital medical safety information reports and to check a report's current status. In preparation for a high volume of web server sessions, the system has a multitude of web servers and is designed for load balancing with respect to web server communication. Only those approved through the user authentication server at login are allowed access to the inhospital medical device safety information reporting system. Safe communication serves as the foundation of server-client communication, and a safe server environment is established with a firewall.

The input and output data of the report system are saved in the DB servers. The relational database is the data organization used in the DB server, and since important data, such as patient data and medical device-related reports, are being saved, auto backup functions and data clustering systems are included. Service programs in charge of actual management work for executing various services needed for reporting medical device safety information are run by the management servers.

The development environment for the in-hospital medical device safety information reporting system is shown in Table 1. Microsoft .NET Framework 4.0 and Windows Server 2008 R2 are used as the operating systems of the application server, and Microsoft SQL 2008 R2 is used as the database. As for application technology, Microsoft Visual Studio 2012, WCF, WPF, ComponentOne, Infragistics, and Cristal Report are utilized.

2) Client system

The client's targets are the reporters, mostly medical device users, medical device safety information managers, and determination committee members. When the central server system is accessed from each client PC, the user can log into the reporting system. Different screens then come up depending on the role of each client user. For example, a reporter can report medical safety information and look up data relevant to his or her role. A medical safety information manager can report medical device safety information and look up all details that have been reported. A determination committee member can look up the primary determination results regarding given medical device safety information and write up an opinion. The system architecture is shown in Figure 1.

2. System Implementation

1) Medical device safety information report screen

A reporter can retrieve patient information from the mandatory electronic records and selectively input medical device information, including product name, serial number, usage conditions, cause of incident, follow-up measures, patient's condition, and other actions taken. A manager can verify the input details of the reporter in Figure 2.

2) Medical device safety information status screen

In the screen, for managing the status of medical device safety information reporting, a reporter is able to look up, revise, and delete pending reports that he or she has written in Figure 3. Once a report is registered, it cannot be revised or deleted. A manager is able to look up all reports and select and save them as registered reports. A determination committee member can look up registered reports and write up a primary determination and other opinions.

3. System Evaluation

This study was approved by the Ethics Committee of the Institutional Review Board of Yonsei University Health System Severance Hospital, Seoul, Korea. We performed a retrospective complete enumeration of 190 cases using medical records from 2012 to 2014 reported at Yonsei University Health System Severance Hospital, Korea.

1) System performance evaluation

Medical device safety information reports were analyzed for 190 cases that occurred from July 1, 2012 to May 31, 2013, a period of 10 months prior to the application of the medical device safety information reporting system, and from July 1, 2013 to May 31, 2014, a period of 10 months during which the reporting system was used. Class 1 was defined as medical devices with very little potential risk. Class 2 was defined as potentially low-risk medical devices. Class 3 was defined as medical devices with a serious potential for risk. Class 4 was defined as highly hazardous medical devices.

2) System survey evaluation

The questionnaire was based on the D&M IS Success Model study [9]. Questionnaires were used to measure the effectiveness of the medical device safety information reporting system. There were four areas, namely, quality, user satisfaction, healthcare anxiety, and individual impact. Also, there were 18 variables (ease of use, response time, reliability, understandability, accuracy, timelines, sincerity, right time, satisfaction of technical service, overall satisfaction, user friendliness, positive attitude, work load, time consumption, potential benefits-patient safety & cost saving, usefulness, decision making, positive benefits) in the four areas. The analysis was based on the questionnaire responses of the 15 reporters who submitted reports in both the pre- and post-reporting system periods.

III. Results

1. Effectiveness Evaluation

1) Performance results

Table 2 shows the performance results obtained before and after the system was applied. In terms of the total number of reports, before the medical device safety information reporting system was implemented, 62 reports were submitted on paper, but after the system was set up, this number more than doubled to 128 reports in electronic form.

In terms of itemized reporting, a total of 45 items were reported. Before the system was used, 23 items had been reported, but this increased to 32 items after the system was put to use.

2) Survey results

Table 3 shows the general characteristics of survey respondents. Out of 15 participants, 1 was male (6.7%), and 14 were female (93.3%). The age distribution of the participants was the following: 1 person in their 20s (6.7%), 4 in their 30s (26.7%), 8 in their 40s (53.3%), and 2 in their 50s (13.3%). In terms of education level, there were 2 college graduates (13.3%), 8 master's degree holders (53.3%), and 5 PhDs (33.3%). As for work experience, 2 had less than 10 years (13.3%), 8 between 10 and 20 years (40.0%), 5 between 20 and 30 years (33.3%), and 2 between 30 and 40 years (13.3%). By position, there were 11 nurses (73.3%) and 4 head nurses (26.7%).

The reliability of the questionnaire was determined by analyzing the reliability between the four areas of the reporting system, namely, quality, user satisfaction, healthcare anxiety, and individual impact and the 18 measured variables. With the Cronbach's alpha of the source data and the standardized data all showing a value over 0.7, questionnaire reliability was established.

The results of the descriptive statistical analysis with respect to questionnaire responses related to the reporting system are given in Table 4. All items received a mean of over 3 points, while positive attitude, potential benefits, and positive benefits all exceeded 4 points, each receiving 4.20, 4.20, and 4.13, respectively. Among the items, time-consumption and decision-making had the lowest mean values, each receiving 3.53. Effectiveness was found to be high for system quality and user satisfaction, but it was relatively low for time-consumption and decision-making.

IV. Discussion

This study sought to understand the current legal regulations concerning the management of reporting adverse incidents related to medical devices and, based on this understanding, evaluated the effectiveness of establishing an in-house medical device safety information reporting system. From the performance-based quantitative analysis results, we found that after the reporting system was put to use, the number of reports increased by more than twice that of the pre-system period, and the reported items became more diversified, as the item types increased from 23 in the period before using the system to 32 after the system was put to use.

The results from the actual proof analysis of the user questionnaire showed the mean value exceeding 3 points for all four items, namely, system quality, user satisfaction, healthcare anxiety, and individual impact. The measurement items of user attitude, potential benefits, and positive benefits each received high scores of 4.20, 4.20, and 4.13, respectively, showing high effectiveness, whereas effectiveness was found to be relatively low for time-consumption and decision-making, with each item receiving a mean value of 3.53.

After the in-house medical device safety information electronic reporting system was created, the number of reports on medical device safety information, which had been previously unreported, more than doubled. However, the number of reports still remain substantially smaller than the actual medical device safety incidents occurring in the field.

With patient safety becoming an international issue in recent years, medical institutions are making a greater effort to monitor even minor mistakes and errors related to medicine and to improve patient safety by focusing on a strategy of prevention. Of the various measures to manage and prevent medical mistakes, the one shown to be most effective is the use of an error reporting system that prevents recurrence by reporting errors that have already occurred [10 11]. Among the impeding factors that lower the rate of incident reporting are fear of punishment [12 13 14 15], lack of faith in improvement after reporting [14 15], lack of knowledge about the scope of incident reporting [16 17 18], and the increase of work load and effort resulting from reporting incidents [15].

For the meaningful monitoring of medical device safety information, adequate training and publicity regarding safety information reporting must be provided, and an awareness of patient safety culture needs to be established [19]. In addition, training and publicity must be continuously provided to successfully establish an in-house medical device safety information electronic reporting system. Based on the accumulation of medical safety data and analyses of medical safety information, medical device safety management should be more actively studied in the future.