Journal List > Healthc Inform Res > v.20(3) > 1075701

Nakamura, Yamashita, Tanihara, and Maeda: Effectiveness and Sustainability of Education about Incident Reporting at a University Hospital in Japan

Abstract

Objectives

The aim of this study was to evaluate the effectiveness and sustainability of educational interventions to encourage incident reporting.

Methods

This was a quasi-experimental design. The study involved nurses working in two gastroenterology surgical wards at Fukuoka University Hospital, Japan. The number of participants on each ward was 26 nurses at baseline. For the intervention group, we provided 15 minutes of education about patient safety and the importance of incident reporting once per month for six months. After the completion of the intervention, we compared incident reporting in the subsequent 12 months for both groups. Questionnaires about reasons/motives for reporting were administered three times, before the intervention, after the intervention, and six months after the intervention for both the intervention group and the control group.

Results

For the intervention group, incident reporting during the 6 months after the intervention period increased significantly compared with the baseline. During the same period, the reasons and motives for reporting changed significantly in the intervention group. The increase in reported incidents during the 6- to 12-month period following the intervention was not significant. In the control group, there was no significant difference during follow-up compared with the baseline.

Conclusions

A brief intervention about patient safety changed the motives for reporting incidents and the frequency of incidents reported by nurses working in surgical wards in a university hospital in Japan. However, the effect of the education decreased after six months following the education. Regular and long-term effort is required to maintain the effect of education.

I. Introduction

Incident reporting is used by healthcare workers to disclose adverse events and near misses. Learning from these types of events is essential for improving patient safety and quality of care [1,2,3]. Mandatory reporting about serious adverse events protects society by guaranteeing that appropriate action is taken [2]. The analysis of adverse events is a powerful learning method within healthcare organizations [4]. High reporting rates are generally associated with a safety-focused culture [5,6], and increases in incident reporting are assumed to improve patient safety [7].
It is necessary to establish effective and continuous incident reporting systems in hospitals to promote patient safety. However, it has been reported that voluntary incident reporting systems identify only a small fraction of incidents [8,9,10,11,12]. Research on reporting quality has shown that education reduces the stress and fear of reporting [9,13,14,15]. In addition, providing healthcare staff feedback increases their voluntary reporting [9]. Most studies have not measured the longitudinal effects of such interventions, and the overall effectiveness and sustainability of educational interventions related to voluntary incident reporting has not been fully proven [16,17]. The above research provided and assessed patient safety education only for newly certified physicians; assessing the effectiveness of safety education for nurses was the focus of the present study.
In other studies targeting multiple professional groups, it was found that the majority of adverse incident reports are generated by nurses [10,14]. At Fukuoka University Hospital, approximately 80% of reports were made by nurses and 10% by doctors. Since nurses were the most frequent reporters due to their numbers and job tasks in the hospital, it was important to determine how to improve nurses' understanding and cooperation in order to improve voluntary reporting on patient safety measures. The aim of this study was to assess the effects of educational interventions on nurse incident reporting, examining the reporting rate and changes in nurses' awareness of voluntary incident reporting.

II. Methods

1. Incident Reporting System at Fukuoka University Hospital

At Fukuoka University Hospital, the process of completing an incident report by a doctor or nurse involved using an electronic reporting system with a signature. The incident-reporting format included both structured and free text sections, and the items within the report included title, date, place, severity level, incident details, incident causes, and future measures. The completed reports were not available for public viewing. All incidents reported were confirmed by seven staff members, including the hospital director and staff from the department of patient safety management; reports were used only to improve patient safety, rather than punitive measures.

2. Study Design

We conducted the study in two gastroenterology surgical wards at Fukuoka University Hospital. Both wards were on same floor of the hospital but were physically separated, and different nurses were assigned to each ward. The nurses of one ward were assigned to the intervention group and those of the other ward to the control group in a quasi-experimental design.
We provided education about incident reporting to nurses in the educational intervention group; nurses in the control group did not receive any intervention. The education consisted of 15-minute-long lectures provided once per month for 6 months, from October 2011 to March 2012 (Figure 1). Nurses participated in a monthly meeting on each ward, and education was provided to the intervention group in conjunction with the meeting. This education had five aims 1) to recognize incident reporting as a significant safety issue, 2) to show how submitting reports is beneficial for reporters, 3) to indicate that errors and misses in medical care are not punitive and that incident reporting is a positive measure, 4) to learn how to prepare computerized incident reporting documents, and 5) to recognize incident report contents. The main educator was a chief nurse who was a patient safety supervisor from the department of patient safety management at Fukuoka University Hospital. In the fifth lecture, the hospital director explained the non-punitive nature and the positive benefits of incident reporting. All nurses in the intervention group, except those who were on duty in the ward, attended the lectures provided.

3. Questionnaire

The questionnaire was adapted from one used by Vincent et al. [18]. The questionnaire consisted of the following five items and responses. Item 4 used a 5-point Likert scale [19,20]: (1) Have you experienced an incident in the past 6 months? (no; yes). (2) Did you submit an incident report? (no; yes but not always; yes). (3) Did you submit the report voluntarily or were you asked to do so by others? (voluntarily; asked by others; both). (4) Participants were asked to provide six reasons/motives for reporting voluntarily, which were presented for rating on a 5-point scale (1 = strongly disagree to 5 = strongly agree). (5) Were any items confusing to input? (multiple answers were allowed: title, evaluation of severity level, reporter, date and location, discoverer, patient attributes, attributes of persons concerned, situation, incident details, incident causes, future measures, and explanation to and reaction of patient and family). In the analysis of item 4, ratings of 5 and 4 were considered 'agree' and those of 3-1 were considered 'disagree.'
The questionnaire was completed anonymously. Missing values were excluded from the analysis. By submitting the questionnaire, nurses indicated their agreement to participate in this research; this was explained on each questionnaire. Ethics approval was obtained from the Institutional Review Board of Fukuoka University Hospital.

4. Statistical Analysis

We subsequently investigated the frequency of incident reporting from the two groups after the intervention. To analyze the sustainability of the effect, we separated the data as follows, considering that the total study duration was two years. The baseline period before the intervention was from April 1, 2011 to September 31, 2011; the first 6-month period after intervention (first period) was from October 1, 2011 to March 31, 2012; the second period was from April 1, 2012 to September 31, 2012; and the third period was from October 1, 2012 to March 31, 2013. The questionnaires were administered to nurses in both groups on three separate occasions every six months to investigate changes in their awareness about voluntary incident reporting (Figure 1).
The frequency of incident reporting in each group was evaluated by reporting rate in this study. The definition of reporting rate is the number of incident reports divided by the total number of hospitalized patient days in the ward during each period. The percentage of change in reporting rates was evaluated by the rate of each period divided by the reporting rate of the baseline period for both the intervention and control groups. The median and interquartile ranges of the reporting rates were calculated before and after the educational intervention to measure changes. To analyze the sustainability of the effect, we aggregated four categories: the baseline period and each subsequent 6-month period, respectively. We used the Mann-Whitney U-test to analyze the differences in reporting rates and Fisher exact test for categorical data (i.e., questionnaire data on reasons for voluntarily reporting). Other continuous data were analyzed with t-tests (e.g., responses to item 5 on the questionnaire). Analyses were conducted using SPSS ver. 19 (IBM, Armonk, NY, USA). The significance threshold was 0.05.

III. Results

1. Characteristics of Study Participants

The baseline characteristics of the intervention and control groups are shown in Table 1. There were no significant differences in reporting rates between the groups with respect to age, career length, and gender. Nurses were reassigned in their positions twice a year at Fukuoka University Hospital. In all, 77% of nurses (20/26) at the baseline continued to work in the same ward until the end of the study.

2. Reporting Rate

During the baseline period, the overall rate in the intervention group was 36 reports/6,423 total patient days or 5.6 reports/1,000 patient days. In the control group, the rate was 76 reports/8,414 total patient days or 9.0 reports/1,000 patient days. During the study period after the intervention, the overall rate in the intervention group was 187 reports/20,184 total patient days or 9.3 reports/1,000 patient days, whereas in the control group it was 236 reports/23,700 total patient days or 10.0 reports/1,000 patient days. In the intervention group, the rate during the 18 months after the educational intervention was significantly higher than at the baseline (p = 0.039). In the control group, there was no significant difference in reporting rates between the baseline and the study periods (p = 0.548).
In both groups, all reported events during the total study period included events with minor injuries or no injury. There were no reports of major injury events. In the intervention group, 51 of 187 (27%) transient moderate injuries were reported with 136 (73%) no injury or others during the study period. In the control group, 39 of 236 (17%) transient moderate injuries were reported with 197 (83%) no injury or others.
Trends in reporting rates as a percentage of change relative to the baseline in each period are shown in Figure 2. Compared with the baseline period, the reporting rate in the intervention group tended to increase in the first period (p = 0.055) and significantly increase in the second period (p = 0.037). However, in the third period, the difference in reporting rates was not significant (p = 0.337). In the control group, there were no significant differences (first period, p = 0.522; second period, p = 0.873; third period, p = 0.521).

3. Questionnaire Results

In the intervention group, there were 26 nurses at the baseline period and during the first period after intervention. Thereafter, the ward arrangement changed and the number of nurses increased to 29 for the second period. Similarly, in the control group, the number of nurses at baseline (26) increased to 27 for the first period after intervention and 31 for the second period.
Table 2 shows the reasons/motives for reporting voluntarily. In both groups, most nurses gave responses, such as 'duty,' 'self-improvement,' and 'improve patient safety in hospital.' In the intervention group, the change in baseline response to the statement 'colleagues are positive' showed an increase for the first period (p = 0.075) and a significant increase for the second period (p = 0.001).
The number of input items that participants found confusing for the 6-month periods before and after the intervention is shown in Figure 3. In the intervention group, the number of confusing items did not change significantly in the first period (p = 0.905) compared with the baseline period, but it significantly decreased in the second period (p = 0.026). In the control group, compared with the baseline period, the change was not significant for the later periods (first, p = 0.535; second, p = 0.893).

IV. Discussion

This study determined that the rate of reporting adverse incidents increased following the 6-month educational intervention, and that nurses who received the intervention became more knowledgeable and more positive about incident reporting. We found, however, that the sustainability of the effect of the educational intervention was limited.
Our results demonstrated an increase in the number of incidents reported in response to brief education. Other educational intervention studies have used several forms of intervention, such as newsletters and information pamphlets [7,14,15] or substantial education sessions (e.g., a plenary day) [17]. In our study, the short 15-minute education provided the information easily in an actual medical setting. Agreement to the questionnaire item 'colleagues are positive' clearly increased as a result of the educational interventions. This indicates a greater awareness of reporting, and many ward staff members developed a more positive attitude towards incident reporting.
The nurses' reasons for reporting voluntarily were elucidated. Most nurses in both groups thought that they had a duty to report at all points, including during the baseline period, suggesting that an understanding of the importance of reporting was already established at our hospital. Similarly, the items 'self-improvement' and 'improve patient safety in hospital' as reasons for reporting were significant in both groups. The nurses expressed positive attitudes towards incident reporting as a way to improve safety.
Previous research has suggested that the main reason for under-reporting adverse incidents has been a lack of knowledge about the methods of reporting and the meanings of the terms used [8,10,11]. In this study, the number of input items that participants found confusing seemed to decrease as a result of the monthly education, suggesting an improvement in participants' understanding of incident reporting. Our electronic reporting system is easy and useful to use; however, there may be further improvements to the system.
To maximize the usefulness of reports, it is necessary to provide some protection to those who voluntarily report incidents, because one reason healthcare workers do not tend to report incidents is a concern about subsequent poor evaluation and fear of disciplinary action [8,9,21,22,23,24]. In the present study, the director of our hospital participated as an educator and explained that reporting errors was non-punitive and that voluntary reporting would lead to positive evaluations.
In our study, incident reports by nurses in the gastroenterology surgical wards increased following the educational intervention, possibly because they learned the meaning and methods of reporting. Our finding is consistent with other research in which knowledge correlated significantly with attitude and practices among healthcare workers [25]. Another factor that possibly contributed to the increase could be the hospital director's emphasis on the positive evaluation that voluntary reporting engenders and the assurance that errors in reporting are considered in a non-punitive manner. This might have had a direct effect on the reduction of nurses' feelings of anxiety about incident reporting.
However, it remains unclear whether the intervention succeeded in increasing the true percentage of reporting, since the number of adverse incidents that actually occurred was not verified during the study. Given the literature regarding under-reporting [8,10,12,13], we believe that the increased reporting rate is more likely due to an increase in the willingness to report incidents, rather than simply an increase in actual incidents. The effect of increasing incident reports is widely expected to improve patient safety [7]. Our study shows evidence of improvements in knowledge of and attitude towards patient safety among nurses as a result of the current educational intervention. Quality and Safety Education for Nurses (QSEN) provides structured education of health professionals to ensure the knowledge, skills, and attitudes required for safety [26]. Patient safety education can have positive effects on knowledge, skills, and attitudes and influence incident reporting [17]. Our study is consistent with QSEN education.
The effect of the educational intervention on reporting rates may decrease over time. In other studies that used educational interventions to improve the reporting of adverse drug reactions, the duration of the change in actions was reported to be 1 year in one study [15] and 16 months in another [27]. In one study, the reporting system registered a 130% increase in the number of reported incidents by residents eight months after the first educational course had started, compared to eight months prior to the first course meeting. This increase remained stable over a further period of eight months, but long-term effects have not been shown [17]. In our study, the effect on the reporting rate continued for 6-12 months after the intervention (i.e., the effect was sustained for six months after the 6-month education period). However, there was no longer a significant increase in reporting rate 13-18 months after the intervention, showing an attenuation of the educational effect. One possible reason for this attenuation is that some nurses who had not received the educational intervention had been assigned subsequently to the intervention ward. For the improvement in reporting rates to continue, repetition of the educational intervention would be needed. The effects of any educational intervention tend to attenuate, and this holds for patient safety education in general. If such education was brief and incorporated into healthcare workers' daily routines on a regular basis, then its effects might be sustained for longer.
Our study design had many positive points. It was a prospective design, able to provide information, while the use of a control group served to prevent confounding associated with seasonal variation, such as nurse reassignment. Both the intervention and control groups worked in gastroenterology surgical wards, and the participants were well matched in terms of background. At baseline, there were no significant differences between the groups with respect to age, career length, and gender that could have influenced the results.
Nevertheless, our study has certain important limitations. First, the study designs of previous studies were a cluster-randomized controlled trial [15] and a prospective multicenter observational study [27]. Our study was a quasi-experimental design. Also, the sample was small. In addition, the baseline reporting rates differed between the two wards, with nurses in the control ward tending to report more incidents. Nurses were not randomly assigned to wards, and the control group may have been more aware about the need for and importance of incident reporting. Another possible confounding factor was that the two wards were on the same floor of the hospital. Nurses from both groups may have shared information with each other; thus, we may have underestimated the overall effect of the educational intervention. Finally, during the study, we presented data only for events with minor injuries or no injury, as reports about major injury events were not submitted during this study period. However, minor event reports are important in helping to identify methods of preventing major adverse events before serious patient harm occurs.
We provided evidence that patient safety can improve after educational intervention. We found, however, that the sustainability of the effect of educational intervention was limited. We recommend that patient safety education in hospitals be provided at regular intervals in the form of brief sessions suited to the daily routines of health professionals. The tendency for educational effects to attenuate suggests that such intervention should be provided continually. Future studies should also make a broader assessment of the possible long-term effects of education on incident reporting.

Figures and Tables

Figure 1
Flow of this study. *Lectures to nurses for 15 minutes once every month during 6 months. The questionnaires were administered during the last week of each period: baseline, end of September 2011; 1st period, end of March 2012; 2nd period, end of September 2012.
hir-20-209-g001
Figure 2
Trends in reporting rates by 6-month period after intervention as percentage change relative to baseline. *Mann-Whitney U-test.
hir-20-209-g002
Figure 3
Number of confusing input items by 6-month period before and after intervention. *t-test.
hir-20-209-g003
Table 1
Baseline characteristics of nurses
hir-20-209-i001

Values are presented as mean (range) or number (%).

at-test, Fisher exact test.

Table 2
Reasons/motives for reporting voluntarily by 6-month period before and after intervention
hir-20-209-i002

aFisher exact test.

Notes

No potential conflict of interest relevant to this article was reported.

References

1. Leape LL. Error in medicine. JAMA. 1994; 272(23):1851–1857.
crossref
2. Kohn LT, Corrigan J, Donaldson MS. To err is human: building a safer health system. Washington (DC): National Academy Press;2000.
3. Reason J. Human error: models and management. BMJ. 2000; 320(7237):768–770.
crossref
4. Vincent C, Taylor-Adams S, Chapman EJ, Hewett D, Prior S, Strange P, et al. How to investigate and analyse clinical incidents: clinical risk unit and association of litigation and risk management protocol. BMJ. 2000; 320(7237):777–781.
crossref
5. Naveh E, Katz-Navon T, Stern Z. Readiness to report medical treatment errors: the effects of safety procedures, safety information, and priority of safety. Med Care. 2006; 44(2):117–123.
6. Hutchinson A, Young TA, Cooper KL, McIntosh A, Karnon JD, Scobie S, et al. Trends in healthcare incident reporting and relationship to safety and quality data in acute hospitals: results from the National Reporting and Learning System. Qual Saf Health Care. 2009; 18(1):5–10.
crossref
7. Mandavia R, Yassin G, Dhar V, Jacob T. Completing the audit cycle: the impact of an electronic reporting system on the feedback loop in surgical specialties. J Healthc Qual. 2013; 35(6):16–23.
crossref
8. Uribe CL, Schweikhart SB, Pathak DS, Dow M, Marsh GB. Perceived barriers to medical-error reporting: an exploratory investigation. J Healthc Manag. 2002; 47(4):263–279.
crossref
9. Evans SM, Smith BJ, Esterman A, Runciman WB, Maddern G, Stead K, et al. Evaluation of an intervention aimed at improving voluntary incident reporting in hospitals. Qual Saf Health Care. 2007; 16(3):169–175.
crossref
10. Sari AB, Sheldon TA, Cracknell A, Turnbull A. Sensitivity of routine system for reporting patient safety incidents in an NHS hospital: retrospective patient case note review. BMJ. 2007; 334(7584):79.
crossref
11. Christiaans-Dingelhoff I, Smits M, Zwaan L, Lubberding S, van der Wal G, Wagner C. To what extent are adverse events found in patient records reported by patients and healthcare professionals via complaints, claims and incident reports? BMC Health Serv Res. 2011; 11:49.
crossref
12. Hwang JI, Lee SI, Park HA. Barriers to the operation of patient safety incident reporting systems in Korean general hospitals. Healthc Inform Res. 2012; 18(4):279–286.
crossref
13. Welsh CH, Pedot R, Anderson RJ. Use of morning report to enhance adverse event detection. J Gen Intern Med. 1996; 11(8):454–460.
crossref
14. Evans SM, Berry JG, Smith BJ, Esterman A, Selim P, O'Shaughnessy J, et al. Attitudes and barriers to incident reporting: a collaborative hospital study. Qual Saf Health Care. 2006; 15(1):39–43.
crossref
15. Figueiras A, Herdeiro MT, Polonia J, Gestal-Otero JJ. An educational intervention to improve physician reporting of adverse drug reactions: a cluster-randomized controlled trial. JAMA. 2006; 296(9):1086–1093.
crossref
16. Jansma JD, Zwart DL, Leistikow IP, Kalkman CJ, Wagner C, Bijnen AB. Do specialty registrars change their attitudes, intentions and behaviour towards reporting incidents following a patient safety course? BMC Health Serv Res. 2010; 10:100.
crossref
17. Jansma JD, Wagner C, ten Kate RW, Bijnen AB. Effects on incident reporting after educating residents in patient safety: a controlled study. BMC Health Serv Res. 2011; 11:335.
crossref
18. Vincent C, Stanhope N, Crowley-Murphy M. Reasons for not reporting adverse incidents: an empirical study. J Eval Clin Pract. 1999; 5(1):13–21.
crossref
19. Russell J, Hollander S. A biology attitude scale. Am Biol Teach. 1975; 37(5):270–273.
crossref
20. Lovelace M, Brickman P. Best practices for measuring students' attitudes toward learning science. CBE Life Sci Educ. 2013; 12(4):606–617.
crossref
21. Barach P, Small SD. Reporting and preventing medical mishaps: lessons from non-medical near miss reporting systems. BMJ. 2000; 320(7237):759–763.
crossref
22. Firth-Cozens J. Barriers to incident reporting. Qual Saf Health Care. 2002; 11(1):7.
crossref
23. Coyle YM, Mercer SQ, Murphy-Cullen CL, Schneider GW, Hynan LS. Effectiveness of a graduate medical education program for improving medical event reporting attitude and behavior. Qual Saf Health Care. 2005; 14(5):383–388.
crossref
24. Kim J, Kim S, Jung Y, Kim EK. Status and problems of adverse event reporting systems in Korean hospitals. Healthc Inform Res. 2010; 16(3):166–176.
crossref
25. Yap J, Lee VJ, Yau TY, Ng TP, Tor PC. Knowledge, attitudes and practices towards pandemic influenza among cases, close contacts, and healthcare workers in tropical Singapore: a cross-sectional survey. BMC Public Health. 2010; 10:442.
crossref
26. Cronenwett L, Sherwood G, Barnsteiner J, Disch J, Johnson J, Mitchell P, et al. Quality and Safety Education for Nurses. Nurs Outlook. 2007; 55(3):122–131.
crossref
27. Tabali M, Jeschke E, Bockelbrink A, Witt CM, Willich SN, Ostermann T, et al. Educational intervention to improve physician reporting of adverse drug reactions (ADRs) in a primary care setting in complementary and alternative medicine. BMC Public Health. 2009; 9:274.
crossref
TOOLS
Similar articles