PDF
ePub
Citation
Print
INTRODUCTION
A journal club can be defined as a group of post graduates who meet regularly to discuss papers in current medical literature.1 The three most common goals for journal clubs are to teach critical appraisal skills, to have an impact on clinical practice and to keep up with current medical literature.2 Among the mentioned goals, teaching critical appraisal skills for reading papers among the residents is more important. This is why, learning critical appraisal skills empower them for distinguishing low vs. high quality articles to improve the quality of care in their patients.3 The Journal club sessions in most clinical departments in Iranian teaching hospitals are conventionally composed of brief summaries of recent papers by residents followed by comments by faculty. The papers presented in these journal clubs are usually selected by the faculty. One of the most important drawbacks in conventional journal clubs is the lack of an ability to increase the critical appraisal skill among residents. Evidence-based medicine (EBM) is a well-known paradigm and has been applied for medical practice for the past two decades.456 In this paradigm, the role of clinical epidemiology, research, and biostatistics in physician performance has been emphasized.78910 Although many medical educationalists have accepted EBM as the gold standard, it is still unclear whether this approach to teach critical appraisal skills is superior to conventional method.11 Many studies have evaluated the satisfaction of residents regarding the EBM approach to journal clubs sessions, 12 but few studies have been conducted regarding evaluation of EBM approach to journal club sessions in promotion of critical thinking among medical students.1113141516 The current study was performed to compare the evidence-based medicine vs. conventional approach to journal clubs sessions in teaching critical appraisal skills to emergency medicine residents.
MATERIALS AND METHODS
1. Research design
This was a double cutoff discontinuation regression quasi-experimental study, performed among emergency medicine (EM) residents of Shahid Beheshti University of Medical Sciences, in Tehran-Iran. The double cutoff discontinuation regression quasi-experimental methodology was applied in the current educational study to adopt educational justice and ethics issues.
1) Study subject
Twenty five EM residents participated in evidence based journal clubs as an experimental group and the same number of residents participated in conventional journal club as control group.
2) Instruments and educational intervention
Two study groups participated in one-hour monthly journal club sessions for six months. Attendance in all journal club sessions was mandatory for all of the participants. The Journal club sessions in the conventional approach were composed of a brief summary of recently published papers by presenting residents followed by comments by the faculty (2-3 random rather than patient case-based papers were chosen by the faculty for each session). Discussion of the quality of the evidence presented was not systematic and consensus regarding the evidence was reached by group opinion. The presenting resident before presenting, did not receive any preparation by the faculty.
Every month, the process of the journal club sessions in the EBM approach was organized in three steps. First, the presenting resident was asked to identify a clinical question derived from an actual patient case and to plan the search strategy in the pub med. In the next step, the presenting resident found one journal article that best addressed the question. And finally in the journal club session, the presenting resident discussed the selected article as a format like the APC journal club17 and the presenting resident and all conference attendees used structured checklist to guide their critical appraisal of the article. These checklists were selected from the web according to the type of study. All of the above mentioned steps in one journal club session were closely supervised by one of the investigators. Before each above mentioned steps, the presenting resident received 60 minutes of direct education from the faculty (the first author) before and after the study, The Fresno Test (FT) was used to evaluate EBM knowledge of the residents who took part in the study.18192021 Also, before and after the study, all participants were examined by analysis of a fictitious manuscript as Critical Appraisal Skill Test (CAST) to evaluate their competency using appraisal skills to review a paper. The title of the factious manuscript was "Comparison of intravenous ketorolac and meperidine in the treatment of biliary colic".
The same title was published in the Journal of Emergency Medicine.22 We used this article to make our factious manuscript by transforming its original writing. Similar to the factious study used in previous studies, there were 7 major and 10 minor flaws, in this factious manuscript.12 Major flaws consisted of methodologic errors such as improper randomization that the EBM approach to critical appraisal emphasized. Minor flaws consisted of methodologic elements that EBM approaches were not emphasized, such as the use of visual pain scales.
Residents were also asked to answer 1 question in an essay format: "After critically appraising this article, would you feel convince to apply its results in your practice? Give at least 3 reasons to support this decision." We assigned 2 points for each major flaw and 1 point for each minor flaw identified; an additional 3 points were given for answering the main test question correctly.
Results of the CAST and FT were scored by an independent evaluator (second author), who was blinded to the allocation of participants in the groups of the study and to whether the answers were from the pretest or posttest.
Minimum and Maximum scores in the FT and CAST were 0-124 and 0-27 respectively. The allocation of the participants into the experimental or control groups was according to their CAST scores before the study as follows: the upper 25 percentile and lower 25 percentile scores put in the control and experimental groups, respectively and the other participants were randomly allocated into both groups. Residents in both groups were given a questionnaire asking about previous research experience, and epidemiology training. Mean pretest and posttest scores were calculated for the experimental and control groups. The effects of the intervention were measured by calculating the changes in scores (posttest score minus pretest score) for each resident and then calculating the mean change in the score for each group.
3) Ethical consideration
The study was reviewed and approved by ethics committee of Shahid Beheshti University of Medical Sciences. EBM vs. conventional approaches to journal club session were applied to teach critical appraisal skills among half of the residents as an experimental group and another half as a control group respectively.
RESULTS
A comparison of the baseline characteristics of the residents in the two groups is shown in Table 1. The range of scores in the FT in the experimental and control groups was 10-120 and 12-74 respectively. The range of scores in the CAST in the experimental and control groups was 0-24 and 2-18 respectively. The mean differences in the FT in the experimental and control groups were 51.8±4.7 and 12.4±4.9 respectively. The mean differences in the CAST in the experimental and control groups were 12.5 ±2.5 and 2.7±2.8 respectively.
Table 2 indicates the data regarding Fresno and the critical appraisal skills pre and post tests in both groups under study and the results of ANCOVA for comparing groups. Analysis of covariance was used to assess differences between groups adjusted for baseline measures of response variables. Seeing as there were no significant differences between the study groups, results were adjusted between the comparison groups for a baseline measure comparison.
As demonstrated in Table 2 no Significant differences were observed between the scores of the pre-tests of the FT and the CAST within the groups. However, the scores for each group showed significant differences (p<0.01) in the post-tests which were shown in Table 2. The R square showed that the model fits the data well. There were significant differences between the two groups in both appraisal and Fresno (p<0.0001).
Improvement on scores of critical appraisal and the Fresno tests in the EBM approach was more than those of the conventional approach. Fig. 1 and 2 illustrate the scatter plots of the FT and the CAST results before and after the study. From the fitted lines, the scores of participants with similar baseline scores in the EBM group were higher than conventional group.
DISCUSSION
The emergency room is a place of "unknowns," which requires emergency physicians to use high levels of critical thinking. In residency training programs, the main aim of journal club sessions is to teach critical appraisal skills to the residents, towards distinguishing low and high quality articles, and using the higher quality, up to date knowledge in clinical practice. The present study showed that attending Evidence-Based Medicine journal club sessions had a comparative advantage compared to the conventional approach for teaching critical appraisal skills among emergency medicine residents.
A comprehensive review of the literature on teaching critical appraisal showed that medical students can improve their critical appraisal skills after an educational intervention. A systematic review study showed that students' knowledge regarding critical appraisal following training, increased 25%.23
Evidence-based practice is the integration of the best research evidence with clinical expertise and patient values.24
Evidence-Based Medicine involves the ability to ask a focused, clinical question; find relevant evidence; critically appraise the evidence; apply the evidence while incorporating patient preferences; and evaluate the process.25
Computer technology, information retrieval, research utilization, and critical thinking skills have been identified as essential prerequisites for the evidence-based practice.26272829
The study of Bazarian et al.11 was similar to the present study, but major differences between their applied methodologies affect the validity and reliability of their results. No golden test has been introduced yet for evaluation of the critical appraisal skills of residents. Applying the methodology of the study of Bazarian, to achieve this goal was very creative. Hence, in the current study, the same method was used. In the study conducted by Bazarian et al.11, there was no statistical significant difference in critical appraisal skills between the two groups of (EBM vs. the conventional approach) which may result from the low sample size of the study. Results of the current study were not compatible with those of the above mentioned study. This a point indicated the strengths of our study. According to the design of our study, those who had lower scores in CAST in pre test exam had more of a chance to be allocated into the experimental group in which organized education was considered. So the allocation of the participants within the groups was congruent with educational ethics.
However, our study suffered from social threats including diffusion threat, which was inevitable as in other educational studies.
In a systematic review evaluating effectiveness of educational interventions promoting evidence-based practices, it was found that students' knowledge, skills, attitudes and behaviors improved when teaching was extended clinically whereas the stand-alone teaching only improved the knowledge in the evidence-based practice.30
We may have gained sufficient knowledge to pass a course test, but our ability to apply it to clinical practice might be in doubt. In our study, the results were based on the FT and CAST scores and the EBM performance of the participants in an emergency department was not evaluated. In future studies, it would be better to design such studies to include the evaluation of the ability of participants to apply the results of appraised papers in the clinical practice.
In our study, we did not measure the point of view of the participants about how much they think critically in the clinical practice and we could not exclude the effect of the residents' dispositions to think critically as a major factor in learning critical appraisal skills.
Furthermore, during the study, other educational programs were used among participants such as teaching and assessment methods which did not facilitate critical thinking could be affecting on the results.
CONCLUSION
According to current study, EBM compared to the conventional approach in journal club sessions seems to be more successful in teaching critical appraisal skills among emergency medicine residents. Further studies need to be developed for increasing the external validity of the results of this study.
XML Download