PDF
Citation
Print
INTRODUCTION
Improvements in surgical quality and patient safety are essential aspects of modern healthcare systems. Many countries have national programs to improve surgical quality. The American College of Surgeons (ACS) pioneered the National Surgical Quality Improvement Program (NSQIP), which was established in 1994 by the Department of Veteran Affairs to improve surgical outcomes and quality of care [12]. This program aims to collect extensive clinical data, develop predictive models for postoperative complications, and provide standardized guidelines to enhance surgical outcomes across various institutions [234].
The ACS NSQIP includes more than 700 participating hospitals and collects detailed clinical data on preoperative risk factors, intraoperative variables, and 30-day postoperative outcomes. The program has accumulated data on over 8.7 million postoperative patients, categorized using Current Procedural Terminology codes. Hierarchical logistic regression models were used to risk adjust the outcomes and provide benchmark data to hospitals. Participation in the ACS NSQIP has significantly improved postoperative outcomes, including reduced complications and mortality rates [25].
NSQIP has also influenced surgical quality improvement programs in other countries, such as the National Clinical Database (NCD) development in Japan. NCD covers over 95% of all surgical procedures in Japan and provides a comprehensive collection of perioperative data. This extensive coverage offers real-world data essential for understanding clinical practices and outcomes. NCD has facilitated significant clinical research, as evidenced by the publication of 206 scientific papers and enables detailed medical analysis, helping to identify areas for improvement. Furthermore, it provides recommendations for enhancing surgical quality and patient safety, making it a valuable resource for the Japanese healthcare system [678910].
With advances in artificial intelligence (AI), surgical quality improvement programs have become more sophisticated. Clinical decision support systems (CDSSs) play a critical role in enhancing surgical quality in the era of AI. Utilizing extensive data gathered through nationwide platforms such as NSQIP and advanced machine-learning techniques, CDSS can offer real-time, patient-specific recommendations to help surgical decision-making and optimize patient outcomes [1112131415161718].
However, the development and deployment of a surgical CDSS requires careful consideration of several factors to ensure its effectiveness and generalizability. Several studies have assessed the performance of the NSQIP Surgical Risk Calculator across multiple international cohorts. Noting that, while it maintained good discrimination, recalibration was necessary to account for differences in baseline risk [5681019]. This emphasizes the importance of validating and adapting CDSS to local healthcare contexts and patient populations. Therefore, there are inherent limitations to directly applying CDSS models derived from the NSQIP data to surgical settings worldwide, as the NSQIP database primarily reflects the surgical population and healthcare practices specific to the United States. Consequently, to ensure the optimal performance and clinical utility of CDSSs in global surgical settings, it is essential to develop models that integrate and reflect the unique characteristics and needs of each country [8102021222324].
NSQIP also has limited variables in the preoperative, intraoperative, and postoperative phases, despite its large case volume. The accuracy of predictive models created by machine-learning techniques improves with more features. However, owing to the constraints of a nationwide database, it is difficult to collect a comprehensive set of variables, limiting its suitability for developing predictive models for specific complications.
In the past, manual data collection methods have acted as barriers to the collection of comprehensive medical data. However, the introduction of electronic medical record (EMR) systems has brought innovation to the healthcare informatics field by enabling efficient data extraction and analysis. Furthermore, advancements in computer performance have laid the foundation for projects such as the Korean Medical Information Mart for Intensive Care, which utilizes intensive care unit data to collect, store, and analyze large-scale real-time data [2526].
The current environment allows researchers to collect a wide range of variables without constraints, thereby enabling them to develop predictive models that can outperform those based on a limited variable set of the NSQIP. The increased availability of comprehensive datasets allows for the exploration of factors affecting surgical outcomes, leading to accurate risk models and personalized treatments. This will help healthcare professionals make informed decisions and improve patient care and surgical outcomes.
In contrast to the United States, South Korea’s healthcare system is characterized by a large number of patients, particularly those undergoing cancer surgery, in major medical institutions located in metropolitan areas. This centralization has led to the accumulation of high-quality surgical data through EMR systems, facilitated by qualified surgeons performing a large number of procedures. Furthermore, this concentrated system provides surgeons with sufficient case volumes, enabling them to achieve a fast-learning curve and resulting in superior cancer surgery outcomes compared with those in the United States and other countries [1314].
Despite their importance and favorable environment, there is still a lack of nationwide surgical complication management programs in South Korea and even the basic data required to create a CDSS have not been secured. Additionally, National medical insurance finance consumption in Korea is expected to increase by approximately 4 times by 2030 owing to aging and strengthening of insurance coverage, and personal healthcare insurance costs are expected to double by 2030. National health insurance finances are expected to become a deficit soon. Therefore, reducing the waste of medical resources and efficiently utilizing them through surgical quality improvement programs will become even more crucial [27].
The first efforts to develop a surgical quality improvement program in Korea began with cholecystectomy in 2016 [28]. The Korean Association of Hepato-Billiary-Pancreatic Surgery, an affiliate of the Korean Surgical Society (KSS), conducted a study to assess surgical quality and develop a prediction model for cholecystectomy. A prospective cohort was established using a surgical database system and postoperative variables and outcomes were analyzed, leading to the development of a risk prediction model. However, the model’s expansion to other general surgical areas was limited by the low incidence of primary outcomes and lack of post-modeling feedback.
To address this gap, the KSS initiated the development of the Korean Quality Improvement Platform in Surgery (K-QIPS) program, a nationwide project aimed at reducing complications, mortality, and medical costs by improving surgical quality using CDSS.
OBJECTIVE CORE TASKS OF K-QIPS
The primary objective of K-QIPS is to contribute to national health and the advancement of the healthcare industry in Korea by reducing surgical complications, mortality, and medical resources by enhancing surgical quality. The program aims to achieve this goal by promoting the widespread implementation of a complication management platform, leading to improvements in national life expectancy, reduction in healthcare costs, and increased efficiency in the utilization of medical resources.
To accomplish these objectives, K-QIPS focuses on 3 core tasks:
-
1) AI-based complication prediction platform
· Development of complication prediction techniques: Leveraging the power of deep learning, this task involves the creation of sophisticated algorithms and models to predict surgical complications with high accuracy.
· Clinical Application of AI Platform: Developing and integrating an AI-based platform into clinical settings to enable personalized patient care and proactive management strategies, thereby minimizing the risk of complications.
-
2) Evidence-based feedback to institutions for postoperative complications
· Institutional Benchmarking: A comprehensive benchmarking system should be established to compare complication rates among healthcare institutions, fostering a competitive and quality-driven environment that encourages continuous improvement.
· Complication incidence data: Data on the incidence of individual complications are provided to help institutions identify and address specific areas of concern, enabling targeted interventions and resource allocation.
· Trend Analysis: Conducting longitudinal analyses of complication occurrence trends and identifying risk factors over time, facilitating the development of data-driven strategies for quality improvement and risk management.
-
3) Guidelines and critical pathways for individual surgery:
· Modification of complication risk factors: Institutional feedback should be utilized to identify and modify risk factors associated with surgical complications, ensuring that healthcare providers implement evidence-based preventive measures.
· Development of standard guidelines: Performance analysis and the development of standardized guidelines tailored to each institution to ensure consistency and adherence to best practices in surgical care.
· Complication prevention protocols: Creating and implementing customized protocols for complication prevention considering each institution’s specific needs, resources, and capabilities.
PREPARATION OF THE K-QIPS PROGRAM
The KSS Research Special Committee, the first organization involved in the development of K-QIPS, held a preliminary preparation meeting on December 19, 2022. We focused on diseases that were common, costly, and influential for policy because it is difficult to cover all surgical fields, such as in the United States or Japan. Gastrectomy, colorectal surgery, hepatectomy and liver transplantation, pancreatectomy, and kidney transplantation were selected as the 5 major surgeries among all surgical fields. During this meeting, the Committee established criteria for selecting the host organization for major surgeries. In the second meeting, on January 9, 2023, the Committee rated the potential host organizations based on the established criteria. The third meeting, held on June 14, 2023, announced the mission of the 5 selected host organizations, laying the foundation for the successful implementation of the K-QIPS program.
ARCHITECTURE OF K-QIPS DEVELOPMENT
KSS, with the support of various stakeholders, including the Korean Surgical Research Foundation (KSRF), the Ministry of Health and Welfare, and participating hospitals, is actively engaged in the development of K-QIPS. This collaborative effort aimed to establish a comprehensive surgical quality improvement platform to enhance patient care and outcomes across Korea.
The main platform for the common variables was developed by KSS as the first host organization. Moreover, 5 specific host platforms for each surgical procedure have been developed by the Korean Gastric Cancer Association for gastric surgery, the Korean Society of Coloproctology for colorectal surgery, the Korean Association of Hepato-Billiary-Pancreatic Surgery for hepatectomy and liver transplantation, the Korean Pancreas Surgery Club for pancreatectomy, and the Korean Study Group of Kidney and Pancreas Transplantation for kidney transplantation (Fig. 1).
DATA COLLECTION AND ANALYSIS (DATAFLOW OF K-QIPS)
The K-QIPS collects data on surgical procedures, patient characteristics, and postoperative complications from participating hospitals. Data were entered by trained surgical clinical reviewers (SCRs) through a web-based electronic case report form (E-CRF) system. SCRs receive regular training on data entry methods and periodic data entry quality feedback through audits conducted by the Data Management Committee. Regarding web-based E-CRFs, KSS will be responsible for collecting data through an E-CRF for common postoperative complications such as ileus, surgical site infection, postoperative pneumonia, acute kidney injury, delirium, acute coronary syndrome, stroke, and other common complications. To predict each complication, we conceptualized CDSS and reviewed the known risk factors for E-CRF. For individual surgery-specific complications, each committee organized by the 5 host institutions created CDSS items to predict detailed complications. To create these CDSS, well-known risk factors were added as additional variables to the E-CRF.
In compliance with data and personal information protection laws, each participating institution completed Institutional Review Board (IRB) and Data Request Board (DRB) reviews for data export. Data were anonymized to ensure that only deidentified information was collected by the KSRF. A double randomly set alternative code was employed to ensure the security of private information, such as that of participating institutions, whereby the matching code was encrypted and stored at both the KSRF and platform host institution. The entire procedure complied with the Guidelines for the Processing of Pseudonymous Information revised by the Personal Information Protection Commission in February 2024.
The data will be analyzed using AI-based algorithms to develop complication prediction models and identify the risk factors for postoperative complications. It provides evidencebased recommendations to improve surgical outcomes. The Complication Improvement Feedback System subsequently synthesizes this information, generating feedback to inform and enhance clinical practice at the participating institutions. Finally, the Korea Health Information Service (KHIS) supports and archives all synthesized data, ensuring a secure and comprehensive repository of clinical information (Fig. 2).
In the K-QIPS program, several CDSS are developed and tailored to specific clinical needs using different AI models and computational frameworks. The data collection and analysis platform within the K-QIPS program is based on specific software and tools developed by the data science team and software development team at MDB Inc. and ZeroOneAI Inc., which have been contracted by KSRF. The K-QIPS system consists of disease-specific E-CRFs and a common E-CRF, with the common E-CRF playing a role in data integration. The disease-specific E-CRFs were developed separately by MDB Inc. and ZeroOneAI Inc. for each disease, while the common E-CRF responsible for data integration was developed by MDB Inc.
Traditional statistical models such as logistic regression, Cox proportional hazards model, and Kaplan-Meier survival analysis are used for traditional clinical outcomes such as postoperative complications, reoperation rates, and mortality. When more complex nonlinear interactions are expected, machine-learning techniques such as random forests, support vector machine, decision trees, and gradient boosting machine are implemented to identify complex patterns and relationships.
In addition, to extend the capabilities of CDSS to handle unstructured data, deep learning methodologies such as convolutional neural network and recurrent neural network are integrated to analyze various data formats such as medical images, pathology reports, electrocardiograms, etc. Model selection is driven by the specific clinical question being addressed and the characteristics of each dataset, with the flexibility to apply and integrate multiple methodologies as needed.
DATA QUALITY IMPROVEMENT PLAN AND AUDIT
A Data Management Committee was established under the KSRF to ensure quantitative and qualitative data management. The Data Management Committee comprises an independent chairperson and secretary who do not belong to any of the 5 host institutions, as well as Data Management Committee members from each host institution. The Committee is responsible for addressing issues related to the IRB and DRB at the participating institutions, managing the overall data flow, data entry to export, and conducting regular audits to maintain data integrity.
To control data quality and recruit large-scale data, we grant access to K-QIPS program research outcomes and data utilization, along with priority participation rights in follow-up projects after the completion of this study. These incentives have encouraged active participation from numerous institutions. Each participating hospital receives financial support for SCR and research nurse education, enabling them to organize their surgical data effectively.
Regarding the standard definition of complications, we initially determined this in the general coordinating committee meeting. Subsequently, we required expert advice from each division’s cardiologists, neurologists, and specialists for specific complications, such as cardiovascular and neurologic complications. After that, we created a definition document for complications and shared it with each SCR.
Through workshops organized by the Data Management Committee, we repeatedly provided online and offline training to the SCRs responsible for inputting complication data. We also have implemented a 3-stage on-site audit process to verify data integrity, demonstrating our commitment to building a large-scale, high-quality medical database.
EXPECTED OUTCOMES AND MILESTONES OF THE K-QIPS PROGRAM
The K-QIPS program will achieve milestones throughout the 6-year study period. In the first year (2023), the successful development of Case Report Forms (CRFs), data entry guidelines, and a web-based electronic CRF (e-CRF) system will lead to the creation of an integrated database for postoperative complications. In the second year, a comprehensive postoperative complication big data platform will be developed, and retrospective data collection will be initiated. By the end of the third year, the program will be completed with the initial establishment of an integrated database of common complications, accumulating data from over 219,200 cases.
In the 4th year, we will focus on analyzing the risk factors for complications, developing surgical critical pathway, and designing optimal CDSS to minimize complications. Prospective data collection will begin this year and continue until the final year of the program, with the aim of identifying over 62,400 cases.
In the 5th year, prospective data collection will continue, and a web-based feedback system will be implemented to provide individual institutions with real-time data on their surgical performance and complication rates. This allows them to identify areas for improvement and track their progress over time. Finally, educational systems aimed at decreasing surgical complications will be designed, and through this feedback, the quality of surgeons at each institution will be enhanced.
In the 6th and final years, a performance analysis will be conducted for each institution, introducing the predictive model and allowing for targeted improvements. Standard guidelines will be established for each institution to ensure a consistent approach to the prevention and management of complications. Finally, an annual report on the nationwide complication status is produced, providing valuable insights into the program’s impact and future direction.
COMPARISON WITH SURGICAL QUALITY IMPROVEMENT PROGRAM IN OTHER COUNTRIES
Table 1 compares surgical quality improvement programs in various countries, highlighting their scope, data collection methods, risk-adjustment techniques, and measured outcomes. K-QIPS integrates AI-based algorithms and multivariate models for risk-adjustment, providing a sophisticated approach to predicting surgical outcomes. Data were collected using a web-based E-CRF by trained SCRs and surgeons to ensure high accuracy and consistency. This advanced data collection method combined with AI provides a significant advantage in the precision and reliability of the predictive models used. Furthermore, K-QIPS measures a broad range of outcomes, including postoperative complications, 30-day mortality, and various CDSSs, to help surgeons.
CLINICAL IMPORTANCE AND FUTURE DIRECTION OF K-QIPS
South Korea has demonstrated excellent outcomes in various cancer surgeries, and the presence of concentrated cases and large-scale centers have allowed high-quality surgeons to report outstanding research results. Although individual researchers at several large-scale centers have conducted multicenter studies, no national-level project has been reported.
K-QIPS represents a significant advancement in the field of surgical quality improvement in Korea. Using extensive clinical data, advanced AI algorithms, and the expertise of specialized surgical societies, K-QIPS aims to reduce surgical complications, mortality, and healthcare costs. This comprehensive platform addresses a critical gap in the Korean healthcare system.
A core strength of K-QIPS is its structured approach to data collection and analysis. The development of a web-based e-CRF system ensured standardized data entry for common postoperative complications, enabling the collection of high-quality, consistent data across participating institutions. This is complemented by a specific CDSS developed by each surgical specialty committee, which is tailored to predict and manage complications unique to the field. This dual approach of common and specific data collection ensures a comprehensive understanding of the surgical outcomes and the factors influencing them.
Another notable strength of the K-QIPS program is its emphasis on compliance with data protection laws and anonymization of data prior to analysis by the KSRF. This approach to data management is essential for protecting patient privacy while allowing for the potent analysis necessary to develop effective CDSS models. The implementation of regular audits and quality feedback further enhances the reliability and validity of the collected data, ensuring that the insights derived from the analysis are based on accurate and representative information. This commitment to data protection and quality assurance guarantees high standards for future national projects.
K-QIPS also emphasizes the need for localized CDSS models. Although potentially accurate in the United States, the NSQIP Surgical Risk Calculator may not be applicable in Korea or other countries. This limitation can be attributed to several factors that influence patient outcomes, including differences in patient demographics, healthcare systems, surgical techniques, and cultural factors. K-QIPS ensures more accurate and relevant risk assessments and recommendations by developing predictive models that reflect local clinical practices and patient demographics.
However, the success of K-QIPS depends on several factors. The engagement and cooperation of all participating institutions are important for the consistent and accurate collection of data. Furthermore, the implementation of CDSS and feedback systems must be user-friendly and integrated into existing clinical workflows to ensure widespread adoption. Ongoing support and funding from government and healthcare organizations will also be essential to sustain the program and achieve its long-term objectives.
LIMITATIONS AND CHALLENGING ISSUES
The initial 3-year retrospective data collection phase may present challenges, such as missing data or difficulties in retrieval, despite the best efforts to gather comprehensive information. Consequently, significant resources may be required during the data preprocessing stages to develop an accurate CDSS, and the actual implementation in the clinical setting may be more challenging than expected. Nevertheless, as most data are concentrated in large centers with their own data management systems, such as Clinical Data Warehouses, the collected data are expected to be representative of the domestic healthcare landscape and effectively reflect real-world situations.
However, the latter 3-year period will involve prospective data collection, which is expected to yield higher-quality data. This approach will facilitate the development of a more accurate model than the existing CDSS and contribute to the improvement of surgical quality. The prospective data collection phase is expected to reduce the limitations of retrospective data and enhance the accuracy and reliability of CDSSs, ultimately leading to improved surgical outcomes and patient care.
Another limitation is that K-QIPS data’s prioritization of personal information protection through anonymization limits its integration with existing public databases in Korea, such as the National Cancer Registry, National Health Insurance Service, Korea Network for Organ Sharing, and the Health Insurance Review & Assessment Service database, and K-Cure service. However, Korea’s established national-level healthcare data integration services offer opportunities for collaboration during the prospective data collection phase. By utilizing these services while ensuring personal information protection, researchers can conduct higher-quality research and more accurate CDSS development. The integration of various healthcare databases can contribute to the development of an accurate CDSS that can enhance surgical outcomes and patient care.
CONCLUSIONS
Despite these limitations, K-QIPS is the first national-level surgical quality management program in Korea. Combining advanced AI technology with comprehensive data collection and analysis has the potential to significantly enhance surgical quality and patient outcomes. Each platform provides valuable insights and recommendations for healthcare providers. As K-QIPS progresses through its milestones, it is important to continuously evaluate its impact and adapt strategies to address the emerging challenges and opportunities for surgical quality improvement. Through the successful implementation of K-QIPS, a novel standard for surgical quality improvement can be achieved, offering a roadmap for other countries to enhance the quality of their healthcare systems.
XML Download