PDF
Citation
Print
INTRODUCTION
This year marks the 63rd anniversary of the Korean Neurosurgical Society (the society). Over the years, our society has produced around 3400 neurosurgeons, established 12 divisions, and eight regional branches. Not only have we seen quantitative growth, but also diverse surgical techniques and the development of new diagnostic equipment have led to the exploration of new clinical areas. The quality of academic research has also significantly improved. Furthermore, with nearly 90 training hospitals entrusted with the training and education of residency in neurosurgery, we are shaping the future of neurosurgery.
Despite the significant progress and changes our society has undergone in the past 60 years, the regulations for neurosurgical training and the evaluation methods for training hospitals have remained largely unchanged for a long time. Especially since the policies of reducing the number of residents that have been in effect since 2013 and the enactment of the Special Act for Resident Training in 2017, the training and medical environments have undergone significant changes. This has raised the need for a comprehensive revision of the regulations, guidelines, and evaluation criteria for neurosurgical training and training hospitals.
We must no longer recognize residents as cheap laborer but as learners who need education. We should create an environment in training hospitals where residents can participate in surgery, education, and research in a balanced manner. In a survey conducted among neurosurgery residents in Korea, this demand for change and innovation can also be confirmed [2].
To achieve this goal, it is necessary for the society to establish minimum competency standards in various areas of training hospitals. For this purpose, the Korean Neurosurgical Society has formed a team, known as the Training Status Investigation Standard Change Task Force (TF team), under the Training Education Committee. We aim to report the results of approximately 2 years of activities, from preliminary investigations to the revision of resident training regulations and changes in training status investigation items.
METHODS
Issues with the current training status investigation
The current training status investigation standards are categorized into clinical practice, education, and research, with 52 points allocated to clinical practice, 30 points to education, and 18 points to research out of a total of 100 points. Within the clinical practice category, apart from inpatient and surgical standards, the remaining 25 points are allocated to facilities, equipment, and medical records. Most hospitals receive the full 25 points in these areas, resulting in very weak discriminatory power in evaluations among hospitals. Therefore, adjustments in the allocation of points in the clinical practice category are necessary.
The allocation of 15 points for inpatient is excessively high, and main surgeries only include vascular and tumor surgeries, making them relatively overvalued compared to other types of surgeries. Additional definitions for main surgeries in areas other than vascular and tumor surgeries are needed.
Although the society has revised the annual resident training curriculum to ensure high-quality training in a rapidly changing training environment, the goals and final competencies of the annual resident training curriculum are not reflected in the evaluation of training hospitals (Table 1). Therefore, new evaluation criteria that reflect the annual resident training curriculum and final competencies are needed.
Activities of the Training Status Investigation Standard Change Task Force (TF team)
To address these issues, the society established the Training Status Investigation Standard Change Task Force (TF team) in January 2021. We held a total of nine team meetings, three workshops for all members of the Training Education Committee of the society, and one joint workshop with responsible supervising neurosurgical specialist at training hospitals. Additionally, we conducted three education sessions for supervising neurosurgery specialists to gather input from society members.
We also sent official documents to 12 division societies to collect diverse opinions, benchmarked training status investigation items from other societies. We also analyzed encrypted data from the past 5 years of training status investigations and the past 3 years of surgical statistics for neurosurgical training hospitals.
RESULTS
The direction of the new revisions includes 1) reflecting the goals and final competencies of the residency annual curriculum, 2) defining core competency surgeries in various areas such as stroke, head injury, spine, and peripheral nerve areas for evaluation, 3) adjusting the scoring system by redistributing points for criteria with insufficient differentiation, 4) specifying the minimum requirements for obtaining eligibility to apply for resident allocation, and 5) introducing a bonus point system reflecting recent trends in the medical field. Ultimately, our aim is to create an evaluation system that encourages balanced participation in clinical practice, education, and research by offering training in various areas.
Revisions
Clinical practice
The existing surgical index score consisted of main surgeries (15 points) and major surgeries (10 points) for a total of 25 points. We reduced the score for main surgeries to 10 points, introduced core competency surgery standards, allocating 5 points each for brain core competency surgery and spine and peripheral core competency surgery, for a total of 10 points. We also adjusted the major surgery score to 13 points, expanding the total surgery index score to 33 points. Previously, only vascular and tumor surgeries were considered main surgeries, but we introduced additional definitions for main surgeries in the areas of spine, pediatrics, and functional surgery. The previous equipment score was 17 points, but as most training hospitals achieved this score, we reduced it to 9 points.
We specified minimum requirements for resident allocation eligibility, and if a hospital meets all of these criteria, they become eligible to apply for resident allocation.
We introduced a new bonus point system. Hospitals performing mechanical thrombectomy or stenting more than 50 cases annually receive 1 point, and those performing it more than 25 cases receive 0.5 points. Hospitals performing peripheral nerve surgery more than 50 cases annually receive 1 point, and those performing it more than 25 cases receive 0.5 points.
Education
Previously, the results of the residency mid-term exam were differentially reflected in a relative evaluation, but we changed it to an absolute evaluation and set 60 points as the pass/fail threshold. We also considered the ratio of the number of residents subject to the exam and the number of residents who passed when calculating the score. We reduced the academic activity score from 8 to 6 points and, instead, introduced a new standard for evaluating the resident’s training plan and assigned 2 points.
The research category retained its existing evaluation standards.
Application of 2022 training status investigation data
When evaluated based on the existing training status investigation scoring table, the average total score for 85 training hospitals was 89.4 points. However, when evaluated using the revised standards, the average score decreased to 85.3 points, marking a decrease of 4.1 points. It is noteworthy that the total score declined in 91.8% of all training hospitals. Examining the scores by category, the average score for major surgeries across all training hospitals increased from 9.3 to 10.9 points, while the score for main surgeries dropped from 13.6 to 7.8 points. The average scores for newly introduced brain core competency surgeries and spine and peripheral core competency surgery were 4.0 and 3.9 points, respectively.
Among the 85 training hospitals, 46 hospitals obtained bonus points, with 44 hospitals receiving vascular bonus points and three hospitals receiving peripheral bonus points. Although hospitals could obtain a maximum of 2 bonus points, no hospital exceeded 1 point.
Out of the 85 training hospitals, 74 met the minimum requirements to apply for the allocation of residents. Among the hospitals that failed to obtain the qualifications for resident allocation, the most common reason was the failure to meet the minimum criteria for spine and peripheral core competency surgery, with a total of six hospitals falling into this category. Additionally, four hospitals had a total score of less than 70 points, making it the second most common reason for failing to meet the minimum requirements. The reasons for the 11 hospitals that failed to meet the minimum criteria are listed in Table 3.
DISCUSSION
When comparing the results of the 2022 Training Status Investigation Data with the virtual assessment based on the revised criteria, the average total score for 85 training hospitals was 89.4 points according to the original criteria. However, when evaluated using the revised criteria, the average score decreased to 85.3 points, representing a decrease of 4.1 points. Remarkably, 91.8% of hospitals experienced a decrease in their total scores. Examining the scores by category, the standard score for major surgeries increased from 10 to 13 points, the overall average score for all training hospitals rose from 9.3 to 10.9 points. Conversely, the standard score for main surgeries decreased from 15 to 10 points causing the average score for training hospitals to drop from 13.6 to 7.8 points. The average scores for newly introduced brain core competency surgeries and spine and peripheral core competency surgery were 4.0 and 3.9 points, respectively.
The decline in the average total score in the revised evaluation was an expected outcome due to the significant reduction in the weighting of equipment scores, which had weak discriminatory power in the existing evaluation criteria. Additionally, the introduction of core competency surgery indicators and adjustments to the weighting of major surgery scores contributed to the overall decrease. The standard deviation of the total score in clinical practice and the standard deviation of the total score increased from 3.07 to 5.09 and 7.03 to 8.56, respectively which means that the discriminatory power of the new revision has improved. When considering the goal of creating a differentiation system for training hospitals, this result can be interpreted as relatively successful.
Among all training hospitals, one hospital experienced the most significant increase in total score, which increased by 13.5 points, while another hospital saw the most substantial decrease, with a decline of 16 points. The hospital with the highest score increase performed a wide range of surgeries, including main surgeries, major surgeries, brain core competency surgeries and spine and peripheral core competency surgeries. In contrast, the hospital with the largest score decrease had fewer inpatient cases and fewer surgeries. This trend aligns with the revised criteria’s aim to encourage diversity in the experiences of residents during their training period.
To reflect recent trends in the medical field, enhance competitiveness with relevant societies, and encourage support and investment, bonus points were introduced. Bonus points, up to a maximum of 1 point for endovascular surgery such as mechanical thrombectomy and stenting and a maximum of 1 point for peripheral nerve surgery, were established, allowing each hospital to obtain a maximum of 2 bonus points. While a score of 2 points may seem small out of a total of 52 points in the category of clinical practice, considering the standard deviation of 3.07 points for the existing clinical practice total score and 5.1 points for the revised criteria’s clinical practice total score among all training hospitals, it can be considered an attractive score. In the simulated analysis of the 2022 Training Status Investigation Data, 46 out of 85 training hospitals received bonus points, with 44 hospitals receiving vascular bonus points and three hospitals receiving peripheral bonus points. Although no hospitals obtained more than 1 point in bonus points, it is anticipated that more training hospitals will be able to acquire bonus points once the system is established.
Through a simulated analysis of the 2022 Training Status Investigation Data, we examined the reasons for non-compliance with the minimum requirements for obtaining eligibility to apply for residency positions in 11 training hospitals. The majority of these hospitals narrowly missed the minimum requirements for residency application, making it likely that they can improve their deficiencies with a little effort during the approximately 2-year grace period. For example, among the reasons for non-compliance, the most common was failure to meeting the annual 50 surgery requirement for core competency surgery in spine and peripheral surgery, with six hospitals falling into this category. However, the average number of core competency surgeries in spine and peripheral surgery for the top three hospitals among these six hospitals was 41, indicating that they can likely meet the minimum requirements during the grace period.
The overall decline in the total scores of the Training Status Investigation was the second most common reason for failing to meet the minimum requirements for applying for residency allocation. This total scores also serve as index criteria designated by the Korean Hospital Society Training Evaluation Committee, encompassing the minimum scores for each subspecialty training hospital and the allocation of residents. Initially, concerns arose that more hospitals would fail to meet these index criteria. Fortunately, the simulated analysis revealed that no hospital failed to reach the minimum score for training hospital designation, which is 60 points. Additionally, only four hospitals failed to obtain 70 points which is the minimum score for eligibility to apply for residency allocation, and all four hospitals received 67 points, indicating that this threshold is sufficiently achievable.
Certain training hospitals, specializing in oncological diseases, were able to achieve high scores when applying the original scoring criteria, primarily due to a high volume of main and major surgeries. However, when applying the newly revised scoring criteria, they fell short of meeting the minimum requirements for core competency surgery in spine and peripheral nerve surgery, thereby losing eligibility for applying for residency positions. Up until now, this issue has been addressed through the rotation training of residents to other hospitals. However, as the Training Status Investigation assesses the capabilities of each training hospital, a focus on enhancing capacity to meet minimum requirements independently is required in the future.
So far, the Korean Neurosurgical Society has assigned many residents to hospitals that perform more surgeries, especially cerebrovascular surgeries and oncology surgeries, regardless of the type of surgery. However, there are not many specialists who actually work in cerebrovascular and oncology field after completing the residency training. Residents are important human resources in hospitals, but we must recognize that they are learners who must be trained to become competent specialists who will take on the future of neurosurgery. It is necessary to create a training hospital environment that encourages residents to develop various core competencies through diverse experiences in various areas, and to establish a performance-based evaluation system that ensures residents receive training in a standardized training environment [3]. It is also necessary to develop a systematic competency diagnosis procedure and tools to establish a training system that can improve the effectiveness of education and training by finding out what individual capabilities are lacking and providing a variety of education and training programs that can supplement them [1]. Our society must continue to strive to create a training system that helps residents to grow into excellent specialists no matter where they train, by changing the training status investigation standards, revising resident training regulations, and establishing a systemized annual training curriculum for residents.
The results of the simulation suggest that the proposed revisions to the training status investigation standards may lead to better discrimination among training hospitals, encouraging more balanced participation in clinical practice, education, and research.
However, several issues still need to be addressed. First, we need to establish a system for providing feedback to training hospitals based on the results of the training status investigation. Second, we need to continue monitoring the situation and make further revisions as necessary based on the actual data collected in 2022 and 2023. Third, we need to ensure that these changes are effectively communicated to all members of the society, including supervising physicians and residents.
CONCLUSION
TF team has proposed revisions to the training status investigation standards with the aim of creating a more balanced and differentiated evaluation system for training hospitals. These changes reflect the goals and final competencies of the residency annual curriculum, introduce core competency surgeries, adjust the scoring system, specify minimum requirements for resident allocation eligibility, and introduce a bonus point system.
The results of the simulation suggest that these changes may lead to better differentiation among training hospitals and encourage balanced participation in clinical practice, education, and research. However, further monitoring and adjustments are necessary, and effective communication of these changes to all members of the society is crucial.
We hope that these revisions will contribute to the improvement of the training and education of neurosurgical residents and the overall quality of neurosurgical care in Korea.
XML Download