PDF
Citation
Print
Introduction
Background
The medical educational activities carried out by medical school faculty members are rapidly changing quantitatively and qualitatively. The recent shift of the medical education paradigm to competency-based medical education has required medical schools to organize and operate an educational system that enables individual students to successfully demonstrate their achievement of learning outcomes [1]. Therefore, the role of faculty members has expanded from traditional instructors to curriculum designers, mentors, advisors, and coaches so that learners can demonstrate the intended outcomes throughout the entire curriculum [2,3]. In addition, coronavirus disease 2019 challenged faculty members and students to an online learning environment, which led to a change from in-person education to online or hybrid education, and from large-scale lectures to small-group or individual-based learning [4,5].
Medical school faculty members struggle to carry out multiple duties: research, clinical practice, education, and public service. Despite increasing demands for engagement in education, education still ranks lowest in priority among their duties. This is mainly due to the fact that evaluation of educational performance is less systematic and its standards are less demanding than the evaluation of research performance and clinical practice performance. In order to be promoted, medical faculty members spend more time caring for patients and producing research results, rather than participating in education [6-8].
Efforts are being made to analyze the medical education activities of medical faculty members and to reward them appropriately for their dedication in education. The Association of American Medical Colleges (AAMC) developed a metrics system to measure faculty members’ effort and contributions to medical education and presented a list of educational activities that faculty members could conduct. Additionally, they proposed the concept of the relative value unit, which refers to the value of the weight of the unit of activities relative to the mission or policy of the medical school, to measure educational activities [9].
In Korea, a study has measured the efforts and contribution of medical faculty members in education at a single medical school [10]. The evaluation items and weights vary depending on the medical school’s mission; however, the items for fundamental medical education activities might be similar. Therefore, it is necessary to investigate the common evaluation items and weighting methods of medical education activities.
Objectives
The aim of this study was to investigate the current status of evaluation of educational performance of medical faculty members through an analysis of promotion regulations of medical schools in Korea. Specifically, promotion regulations of each medical school were classified through content analysis, and contents of individual medical school’s promotion regulations were reorganized according to the classification.
Methods
Ethics statement
Ethical approval was not required for this study. This study did not include a clinical trial and did not collect any personal data. Information that was publicly available on websites was collected and analyzed.
Phase 1: Qualitative study
Data collection
In August 2022, we searched the websites of medical schools and their universities in Korea to identify publicly accessible promotion regulations with provisions on medical educational activities. Data were collected from the promotion regulations of 22 out of 40 medical schools (Fig. 1).
Personal characteristics of the research team
The research team consisted of experts on medical education. J.P., a PhD in educational measurement and evaluation, and H.W.J., an MD, PhD in medicine, were professors in the Department of Medical Education at a school of medicine with several years of experience in the development of the faculty member evaluation systems.
Data analysis
Data were analyzed through deductive content analysis. This study defined and categorized the educational activities of faculty members recognized in the promotion regulations of each medical school according to the categories suggested in the AAMC framework to improve the correspondent validity and compared the results between the AAMC and our results [9]. The AAMC framework defines 4 educational activity categories: teaching (T), development of education products (D), education administration and service (A), and scholarship in education (S), and 17 main activities and 69 sub-activities. We added the student affairs (SA) category because many medical schools in Korea recognized work related to student affairs, such as student counseling and career coaching, as educational activities. Educational activities that did not fit into any of these 5 categories were classified as “others.” Two researchers (J.P. and H.W.J.) iteratively read and reviewed the contents in the collected promotion regulations, and codes were assigned to all documents. according to these 6 categories and their sub-activities. We also explored scoring methods of educational activities and factors for adjusting the weight of educational activities. The coding structure that emerged from the data was critically discussed between the researchers until consensus was achieved.
Phase 2: Quantitative study
Variables
The coverage rate of educational activities in each category and the inclusion rate of individual educational activities in promotion regulations were analyzed. The type of institution, the year of establishment, and the number of faculty members at each medical school were variables for the basic characteristics of medical schools.
Data sources/measurement
Main data were the contents of individual medical school’s promotion regulations which were reorganized according to the classification based on the results of the qualitative research (Dataset 1). Data on medical schools’ basic characteristics were collected from the database of the Korean Association of Medical Colleges. Scoring methods and quality evaluation of medical educational activities is available at Dataset 2.
Bias
No selection bias was identified. This study included all medical schools that met the study’s eligibility criteria.
Study size
There are a total of 40 medical schools in Korea, and a survey was conducted on all schools, of which 22 met the research conditions. Therefore, no sample size was estimated for quantitative analysis.
Statistical methods
The quantitative variables were presented as frequency and percentage. Frequency, descriptive, and correlation analyses were performed. Correlations between the characteristics of medical schools and categories of educational activities were assessed using Spearman’s or Pearson’s correlation analysis. All data were processed in IBM SPSS ver. 27.0 (IBM Corp.).
Results
General characteristics of medical schools
Sixteen were private medical schools and six were public medical schools. Eight medical schools had 101–200 faculty members, seven had 201–300 faculty members, and seven had more than 300 faculty members. Ten medical schools were established after 1980, and the remaining 12 were established before 1980 (Table 1).
Educational activities specified in the promotion regulations of Korean medical schools
Classifying medical educational activities stipulated in the promotion regulation of medical schools, the researchers yielded 6 categories and 20 activities with 57 sub-activities; teaching (5 activities and 16 sub-activities), development of education products (3 activities and 9 sub-activities), education administration and service (4 activities and 16 sub-activities), scholarship in education (5 activities and 12 sub-activities), student affairs (3 activities and 4 sub-activities), and others (Table 2). The “others” category had unique activities, such as donations, fundraising, medical students’ pass rate on the Korean Medical Licensing Examination, the pass rate on the specialty examination, disciplinary results, and so forth.
The coverage rate of educational activities in each category was the highest in the development of education products category (2.4/3, 78.8%). The second highest was teaching category (3.6/5, 71.8%), and the lowest was scholarship in education category (0.8/5, 16.4%) (Table 2). The inclusion rate of individual educational activities in promotion regulations was the highest in lecture activities, development of personnel, and direction of education components (21/22. 95.5%) (Table 2).
Correlation and comparison between categories of educational activities and characteristics of medical schools
Private schools had a strong positive correlation with a longer history and the inclusion of medical educational activities in the promotion regulation in the following categories: teaching, scholarship in education, and student affairs. Private medical schools tended to include more educational activities than public medical schools (Fig. 2A, Supplement 1), and this tendency was prominent for scholarship and student affairs. The number of activities in the education administration and service category increased as the number of faculty members increased (Fig. 2B). The “scholarship in education” and “student affairs” had a moderate correlation with the “established year of school.” The correlation between individual activities and characteristics of medical school were shown in Table 3 [13] (Supplement 2).
Evaluation of educational activities
Adjustment factors for the weight of educational activities
Four factors were identified to adjust the weight of medical educational activities; the characteristics of the target subjects, the characteristics of the faculty members, the number of faculty members involved in the activity, and the difficulty of the activity. Each factor, especially the difficulty of the activity, had various weighting methods. Fourteen medical educational activities were evaluated with more than one of these 4 weighting factors. The faculty members number was considered in evaluating lecture activities, small-group activities, and development of educational materials. The faculty members’ characteristics were considered only in evaluating the development of personnel (Table 4).
The scoring method and the quality evaluation
Most medical schools chose a criterion-referenced evaluation, with a scoring method based on raw data or grade scores within the range. For 14 medical educational activities, several medical schools displayed an upper limit on the points that could be received. There were 2 types of penalties: receiving negative points and rejecting performance (conditional approval). Examples of point deduction included failure to submit a lecture syllabus before the lecture, and the examples of rejection of performance included failure to meet the attendance time or frequency, such as participation in faculty member development seminars and committees. There were 3 types of evidence utilized to evaluate the quality of educational performance. Quality evaluation was conducted for 7 educational activities, among which lecture activities were evaluated most frequently (Table 5).
Discussion
Key results
This study provided an overview of the current evaluation status of medical school faculty members’ educational performance by identifying medical education activities and their assessment methods along with weighting factors through an analysis using mixed methods in Korea. The inclusion in the promotions regulations and the weighting and scoring method for the specific activity differed from school to school, which implied that medical schools demanded various educator roles from their faculty members depending on their mission and policies.
Interpretation
While almost all medical schools included lecture activities in their promotion regulations, laboratory activities and clinical practice activities were included in relatively a small number of medical schools. Considering that many faculty members spend a considerable amount of time on them, effort seems necessary to reflect them more actively in the promotion regulations. The high inclusion rate of the development of personnel in promotion regulations seemed to be related to the standard of the Korean accreditation of medical schools such as “at least 50% of full-time faculty members must participate in medical education training or education-related faculty member development programs for at least 3 hours a year” [14].
Activities in the scholarship in education category, except for presentations in education, were rarely included in the regulations. Besides publishing papers, there are many activities related to scholarship in education, which include writing education research reports, making abstract-based presentations at educational academic meetings, giving symposium lectures, and so on. Medical schools should strive to ensure that all educational activities of faculty members are included in the promotion regulations, so that faculty members dedicated to medical education can be properly recognized and evaluated for their effort [6-8].
Among 4 adjustment factors for the weight of educational activities, the difficulty of the activity was the most frequently considered, and the number of items in each factor varied among medical schools. The AAMC recommended implementing quantitative and qualitative weights differently in stage 1, and completing the weights based on the mission of the school in stage 2 [9]. When establishing standards for evaluating faculty members’ educational performance, the weight of educational activities must be determined by reflecting the mission and policy of medical schools so that faculty members can accept the evaluation standards and devote themselves to educational activities consistent with the direction of the mission and policy.
Most educational activities were evaluated with a scoring system in which the more faculty members did, the more points they got. The penalty, either point deduction or conditional approval, was given when the activities were not properly carried out, or when the additional tasks necessary to complete the activities were not met.
Limitations/suggestions
First, the data of this study were based on the promotion regulations published on the websites of medical schools or universities. There was a possibility that some universities/medical schools did not disclose the details of promotion regulations on the web.
Second, the small number of study subjects made further statistics other than correlation analysis impossible. In addition, caution may be required when interpreting the result of correlation analysis with a small sample number. Therefore, it would be appropriate to conduct a more comprehensive study using surveys and in-depth interviews through representative organizations such as the Korea Association of Medical Colleges.
Conclusion
Rapidly changing medical education calls for more efforts of faculty members to meet the needs of learners and society. In order for faculty members to devote themselves actively and passionately to education, they must receive reasonable rewards. This study analyzed the status of medical education activities reflected in the promotion regulations of medical schools in 2022 in Korea. This study provides essential data for improving the reward system for medical faculty members’ educational efforts.
XML Download