PDF
Citation
Print
Introduction
Background/rationale
The Korean Institute of Medical Education and Evaluation (KIMEE) operates the accreditation system of medical schools in Korea, with the goal of supporting continuous improvements in the educational environment and programs. The accreditation standards have changed over the past 20 years (from 2000 to 2019) across the first cycle (2000–2005), the second cycle (2007–2011), the post-second cycle (2012–2018), and the Accreditation Standards of KIMEE 2019 (ASK 2019) cycle. A report was issued on the changes in accreditation standards and development directions of the KIMEE (Supplement 1), which is summarized in this review article.
Objectives
This study aimed to explain the following aspects of the accreditation process: the development process, setting principles and directions, evaluation items, characteristics of the standards, and validity testing in each cycle. It may be worthwhile to create a record of the background and features of each accreditation cycle for the sake of future generations who are involved with the accreditation of medical schools, both in the KIMEE and at various medical schools throughout Korea.
The first cycle (2000–2005)
Development process
To prepare for the first cycle of medical school accreditation, the Accreditation Board for Medical Education in Korea (ABMEK), the predecessor of the KIMEE, formed an Evaluation Criteria Working Committee to develop evaluation accreditation standards in 1998 [1]. The Evaluation Criteria Working Committee of the ABMEK developed draft versions of accreditation criteria for medical universities through meetings, workshops, and opinion-gathering, and these draft criteria were applied to preliminary evaluations of newly established medical schools in 1999. By supplementing the deficiencies identified in the preliminary evaluation, the criteria for the first cycle of medical school accreditation were established, and through this process, 41 medical schools were evaluated over the 5-year period from 2000 to 2004 [2].
Setting principles and directions
The criteria for the first cycle of medical school accreditation were based on the CIPP evaluation model [3], which provides a systematic approach by setting the following 5 principles for evaluation criteria: usefulness, adequacy, clarity, feasibility, and future orientation. These accreditation criteria aimed to present future-oriented considerations in light of the real-world circumstances of Korea, to improve the quality of medical schools and to foster greater social accountability, and to ensure the minimum necessary educational conditions and curriculum. The evaluation areas were divided into mandatory standards and recommended standards. Mandatory criteria were those that each medical school was required to meet for the purposes of basic accountability in medical education. Recommended criteria were future-oriented standards that encouraged medical schools to develop themselves to achieve the recommended levels. The KIMEE took into account the “Rules for establishing a medical school” and “Medical school evaluation standards” proposed by the Korean Council for University Education in 1996 [2]. In particular, mandatory standards related to facilities and working personnel were aligned with the “Rules for establishing a medical school.” The balance between quantitative and qualitative criteria was also considered.
Evaluation items
The evaluation items for the first accreditation were based on the 93 items proposed by the Korean Council for University Education in 1996, excluding graduate schools. Fifty evaluation items were selected in the following 5 areas: educational goals and curriculum; students; professors; facilities and equipment; and administration and finance. Of the 50 items, 18 items were mandatory, and 32 items were recommended (Table 1).
Characteristics of the standards
The educational environment and the entire curriculum of medical schools were evaluated, including the following components: the circumstances under which the curriculum was implemented; the degree to which available resources were invested to achieve the objectives; the interactions among the resources allocated to achieve the objectives; and the achievements of education, research, and community service. The items that were used reflected core elements of medical education. Specific aspects of medical education were also analyzed, including medicine-related interpersonal skills, student research activities, the distinction between the basic and clinical curriculum, and the distinction between basic and clinical teaching. This level of specificity is infrequently addressed in other educational fields. Universities were recommended to provide a plan for improving the quality of medical education based on a long-term vision.
Validity testing of the measurement tool
A study was conducted to examine the classification of validity and predictive validity of the 50 items used as accreditation standards for medical schools in 2000 (Table 1). The validity of the standards for medical school accreditation varied depending on whether schools were classified as leading, average, or unconcerned; therefore, applying uniform standards to all types of schools was concluded not to be a reasonable approach to enhancing the quality of medical education. The standards of accreditation predicted 84.4% of students’ level of satisfaction [4]. Another study surveyed 309 professors to test the validity of the measurement tool, which contained 50 items. In that study, 70.2% of the faculty members who responded stated that the accreditation system achieved its goals, and 80.7% of them pointed out the necessity of a continuing accreditation system. As a result of the 5-year accreditation process, 32 medical schools were accredited and 9 medical schools were accredited on a probationary basis. The satisfaction level for items regarding students and facilities was lower than that for items regarding curriculum and administration/finance [5].
The second cycle (2007–2011)
Development process
With the end of the first cycle of the medical school accreditation project in 2005, opinions were solicited on problems with the standards of the first cycle in order to provide directions for standards to be applied in the second cycle of accreditation. The committee held several meetings and workshops to develop the accreditation criteria for the second cycle, which were revised and supplemented through 2 public hearings in 2005 and opinions from 41 medical schools. A special committee was formed to discuss and address these issues and to increase the reliability of the application and interpretation of standards by linking the accreditation evaluation criteria with the Medical Education Database (Korea Medical School Information System). Through this process, plans for the second cycle of the accreditation criteria were completed in 2007.
Setting principles and direction
The principles of the standards focused on social responsibility and educational excellence. According to the degree of accountability of each university, the criteria were divided into essential and recommended, as well as those focusing on excellence. Essential criteria were related to the performance of universities regarding basic accountability, while recommended criteria were related to improvements in the quality of education. Criteria focusing on excellence were related to the pursuit of excellence in education. Furthermore, to improve the quality of medical education to the international level, international accreditation criteria were satisfied, including the standards of the World Medical Education Association (WFME), the International Medical Education Organization, the American Medical Education Joint Commission, and the British Medical Association.
Evaluation items
In total, 109 evaluation criteria were assessed in the second cycle, including 41 essential criteria, 34 recommended criteria, and 34 criteria related to excellence in the areas of operation of the university system, educational goals and curriculum, students, faculty, facilities and equipment, and graduate education [5,6] (Table 2).
Characteristics of the standards
The evaluation criteria were divided into essential and recommended criteria, as well as criteria regarding excellence. While the importance of quantitative standards was reduced, the importance of qualitative evaluation criteria for judging the actual content and quality of medical education increased. Competence after graduation was introduced as a criterion to be evaluated, as were the structural standardization of university hospitals and graduate school programs [6].
Validity test of the measurement tool
A meta-analysis of items used in the criteria for the second cycle of accreditation found that most of them were important and appropriate, except for a few items. The less critical items were alumni and community standards in the field of university operation, complementary and integrated medicine as educational goals, and areas related to the curriculum. For adequacy, low scores were found for the following items: complementary and integrative medicine in the curriculum area; the proportion of graduates who entered fields other than clinical medicine in the student area; the ratio of basic and clinical medicine faculty numbers in the faculty area; and ensuring places in university-affiliated hospitals, information system operation, and faculty spaces in the facilities area. The validity of the criteria used in the second cycle of accreditation was high. It was suggested that ongoing efforts should be made to improve deficiencies in qualitative aspects that cannot be measured by quantitative standards and to develop accreditation standards that balance the need of respecting diversity and pursuing specialized skills and excellence at each medical school [6].
The goodness of fit was tested for the 109 items used in the second cycle of accreditation. Dichotomous data on 109 items from 40 medical schools were analyzed according to the Rasch model. All items were in the acceptable range in terms of the infit mean square, while 107 items were in the acceptable range for the outfit mean square. These findings meant that 2 items were outliers: one was “the college must have education, research, and patient care policies regarding social accountability and such policies must be practiced” and the other was “the ratio of faculty members who graduated from the same college was 70% or less among the total faculty of the medical school.” Therefore, the items used in the second cycle of accreditation from 2007 to 2011 by the KIMEE were favorable [7].
Post-second cycle (2012–2018)
Development process
In order to eliminate the concept of the cycle, the KIMEE decided to ask medical schools to submit mandatory self-assessment reports every 2 years in the next round of the accreditation process. Furthermore, the KIMEE was recognized as the medical education accreditation agency of South Korea by the Ministry of Education for the 5-year period starting on May 12, 2014, based on the Higher Education Act [8]. Some of the accreditation criteria were revised before and after recognition by the Ministry of Education.
Development of criteria before recognition by the Ministry of Education
Starting in 2010, the KIMEE Standards Committee conducted several meetings and workshops to prepare the post-second cycle accreditation criteria based on the results of a meta-analysis of the previous 2 cycles and opinion-gathering through public hearings. The accreditation criteria were approved in January 2011 and applied starting in 2012.
Development of criteria after recognition by the Ministry of Education
In November 2013, 36 quantitative items were reviewed out of 97 basic standard items according to the request of the Ministry of Education to revise items to be qualitative. Of the 36 quantitative items, 27 items were revised to be qualitative, including 2 sub-areas of the operational system of the university, 5 sub-areas of basic medical education, 2 sub-areas of students, 4 sub-areas of faculty members, 3 sub-areas of facilities and equipment, and 1 sub-area of graduate education. The remaining 9 quantitative items were kept; these items were related to curriculum operation, evaluation, and clinical practice (duration, methodology, etc.) .
Setting principles and directions
To improve the accreditation standards, the following principles were introduced after a review of the results of the meta-analysis of the second accreditation cycle [6]: first, the need to establish the minimum level of the educational environment and educational programs; second, international-level standards; third, standards respecting the specialization and excellence of each medical school as part of pursuing the diversity of medical schools; fourth, long-term, future-oriented standards; fifth, standards to fulfill society’s demands; and sixth, standards for performance-based education.
Evaluation items
There were 140 post-second-cycle evaluation items, including 97 basic items and 43 supplemental items in 6 areas with 20 sub-areas. The number of items in each area remained the same after the KIMEE was recognized by the Ministry of Education, but 27 quantitative items were revised to qualitative items and 1 item in the area of basic medical education was added (Table 3).
Characteristics of the standards
The curriculum based on learning outcomes focused on the qualities that graduates could achieve above a certain level of competence as doctors, reflecting the international trend of emphasizing competence-based education for the development of students’ performance. The qualitative criteria were strengthened with the goal of improving the environment and programs of basic medical education and to protect the right of students to receive a high-quality education and to guarantee the health of the public. The essential items and recommended items of the second accreditation cycle were combined into the basic items, which reflected the steps necessary to ensure the minimum requirements of medical education. In the area of students, the aspects relating to student protection and rights were strengthened [6,8].
Validity test of the measurement tool
A meta-analysis of the post-second cycle accreditation was conducted to determine whether the evaluation items helped universities to develop. In a self-evaluation, 77.8% of faculty members stated that the basic items were helpful, while 52.6% of them reported that the supplemental items contributed to the development of the university [9].
ASK 2019 (2019–present)
Development process
To be recognized by the WFME as a medical school accreditation agency, the KIMEE conducted a self-assessment through expert meetings and workshops starting in 2013. The KIMEE received WFME certification as an accreditation agency in 2015. Furthermore, the KIMEE developed the ASK standards suitable for the circumstances of medical education in Korea based on the WFME evaluation standards. To do so, the KIMEE collected opinions through 3 sets of public hearings in 2015 and 2016. The ASK 2018 criteria were developed through meetings with related institutions such as the Korea Association of Medical Colleges, the Korean Association for Basic Medical Scientists, and the Korean Society of Medical Education. ASK 2019 was announced in May 2017 as a tool to be fully implemented starting in 2019. The KIMEE also developed guidelines for the application of ASK 2019 and announced them in February 2018.
Setting principles and directions
The following directions were set in consideration of the circumstances of medical schools in Korea based on the WFME global standards for quality improvement in basic medical education (2015 revision) [10]. The overall structure and composition were also revised according to the WFME global standards. The basic items were those that universities were required to meet, corresponding to the purpose of accreditation. Supplemental items were desirable, future-oriented standards for universities and basic medical education, aiming to reform medical education to follow international-level best practices. The following principles were followed: First, it was necessary to maintain a balance between the WFME evaluation items and the post-second accreditation items. Second, WFME evaluation items that did not fit the circumstances of medical education in Korea would not be used. Third, some of the WFME evaluation items would be added or revised in accordance with the post-second cycle items according to the circumstances of medical education in Korea. Fourth, any evaluation items not corresponding to the basic medical curriculum would not be included. Fifth, legal requirements specific to Korea would not be included in the evaluation items.
Evaluation items
The ASK 2019 consisted of a total of 143 items, consisting of 92 basic items and 51 supplemental items in 9 areas and 34 sub-areas (Table 4).
Characteristics of the standards
Mission and outcomes
Based on its founding philosophy, each university should establish its mission by actively gathering opinions from stakeholders such as students, faculty members, university staff, headquarters staff, health care workers, and community medical associations. If graduates’ performance can be assessed before the institution’s mission is established, a careful review of the graduates’ performance and the intended educational outcomes is necessary.
Curriculum
The basic medical curriculum area of the post-second cycle accreditation was divided into 3 components: curriculum overview, curriculum development and support, and curriculum composition and operation. The curriculum must be organized based on the institution’s mission and graduates’ achievements. As such, the curriculum must be continuously revised and supplemented to reflect changes in the medical environment. The basic medical curriculum must be linked to a curriculum that incorporates post-graduate education and lifelong learning, and the curriculum should be operated in a way that reflects community opinions and needs. This means that faculty, students, staff, alumni, parents, and health care stakeholders should all participate in training programs for physicians who are able to carry out primary care.
Student assessment
It is proposed that student assessments should be operated systematically. In addition to paper and pencil-based assessments, students’ knowledge, skills, and attitudes are required to be evaluated. An emphasis is placed on student assessments to ensure that they achieve the intended educational outcomes through a framework grounded in performance-based education.
Students
The evaluation items did not change dramatically compared to the post-second cycle accreditation. However, academic guidance for students who do not succeed in passing to a higher grade and students’ participation in medical school committees were changed from supplemental standards to basic standards.
Faculty
Assessment of faculty members’ research performance, including the number of journal articles or amount of research funding, was excluded from the post-second cycle accreditation. Instead, the ASK 2019 criteria evaluate the degree of educational and development support for faculty by medical schools, with an emphasis on the balance among education, research, and social service in faculty members’ performance. These criteria assess whether the composition of faculty members in each curriculum is balanced, whether there is a policy to support all faculty members in understanding the curriculum, and whether the medical school supports faculty members in developing their competencies in integrated education.
Educational resources
This area is characterized by the transition of educational resources to student-centered education. From the student-centered perspective, evaluation items include clinical practice resources, information technology, medical research, educational expertise, and exchange programs. One of the new additions is educational expertise, which refers to physicians, pedagogues, sociologists, and institutions that deal with the processes, practices, and problems of medical education. In recent years, the flow of medical education has required extensive changes throughout the educational process. The demand for development, improvement, and evolution of new curricula is inevitable; therefore, a strategy is needed to solve these problems. This area aims to secure educational expertise and to reflect natural choices for the times.
Education evaluation
An emphasis is placed on expanding the curriculum beyond the elements that are traditionally considered as part of the curriculum in order to facilitate the continuous improvement of education-related activities. In other words, it is necessary to systematically evaluate all courses related to education, including the curriculum as a whole, from the time of enrollment to graduation, through planning, implementation, data collection and analysis, and feedback. Therefore, while the post-second cycle accreditation criteria focused on curriculum evaluation, the ASK 2019 criteria regularly monitor and evaluate all activities related to education, including the curriculum, educational resources, faculty, staff, students, university culture, and learning environment. A systematic implementation is required to ensure a comprehensive evaluation capable of contributing to continuous improvements.
Operational system and administration
Considering the similarities of the ASK 2019 criteria to the post-second cycle accreditation criteria, it is necessary for medical schools to have an administrative system capable of executing the school affairs, budgeting systematically, and maintaining operational standards even if the dean changes.
Conclusion
The KIMEE was certified by the Ministry of Education of the Korean Government as an official accreditor for basic medical education in 2014 in response to ongoing requests by medical educators [11]. Meanwhile, a national, large-scale objective structured clinical examination (OSCE) was introduced as part of the licensing examination to improve practical clinical education in the last 2 years of the medical curriculum in 2010. This was enabled by the accreditation standards that prescribed the installation of skill labs and OSCEs at medical schools [12]. At the WFME Recognition Committee meeting held in Melbourne, Australia, on September 19, 2016, the KIMEE was recognized as the accreditation agency in Korea for 10 years from the time of recognition.
The WFME-centered international standardization of medical education has been developed to ensure the minimum quality of medical practice through a common accreditation system of medical schools. Accordingly, to raise medical education to the international level, evaluation standards corresponding to the international level must be developed. The KIMEE has developed an accreditation system of medical schools in South Korea in a step-by-step manner for the past 20 years. The outcomes were remarkable, as highlighted herein.
Most notably, the ASK 2019 system was established, which is comparable to the top-tier systems throughout the world. It is no exaggeration to state that the history of the development of medical education in Korea during the past 20 years has proceeded hand in hand with the history of the development of the accreditation of medical schools by the KIMEE. Understanding previous processes will be helpful for developing items for the accreditation of medical schools in Korea in the future. Furthermore, the following 5 assessment elements are suggested for the further development of the accreditation process: institution- and culture-based assessments; future-oriented assessments; excellence- and diversity-oriented assessments; qualitative assessments with higher validity and reliability; and authentic assessments.
XML Download