Journal List > J Educ Eval Health Prof > v.12 > 1148938

Yim: Exploration of examinees’ traits that affect the score of Korean Medical Licensing Examination

Abstract

Purpose:

It aims to identify the effect of five variables to score of the Korean Medical Licensing Examinations (KMLE) for three consecutive years from 2011 to 2013.

Methods:

The number of examinees for each examination was 3,364 in 2011 3,177 in 2012, and 3,287 in 2013. Five characteristics of examinees were set as variables: gender, age, graduation status, written test result (pass or fail), and city of medical school. A regression model was established, with the score of a written test as a dependent variable and with examinees’ traits as variables.

Results:

The regression coefficients in all variables, except the city of medical school, were statistically significant. The variable’s effect in three examinations appeared in the following order: result of written test, graduation status, age, gender, and city of medical school.

Conclusion:

written test scores of the KMLE revealed that female students, younger examinees, and first-time examinees had higher performances.

INTRODUCTION

Various factors affect a person’s test results. In evaluating individual achievement, one cannot overlook personal traits and the influence of the groups to which a person belongs. The influence that such factors have on a student’s achievement is a research subject in itself or a variable that needs to be controlled in comparing the performance between groups. Interest in academic achievement can go beyond general results, such as means and deviations, and develop into more specific questions, such as the following: Do male or female students have higher academic performance?; What examinees’ traits are associated with higher test scores? It aims to explore how much influence the variables of individuals or groups such as examinees’ gender, age, graduation status, location of medical school, and test result (pass or fail) have on test score, the relative influence of variables, and the inter-group differences of test score of the Korean Medical Licensing Examination (KMLE) from 2011 to 2013.

METHODS

Subjects and materials

The target of this study was the test results of three consecutive year Korean Medical Licensing Examination from 2011 to 2013. The number of examinees for each examination was 3,364 in 2011, 3,177 in 2012, and 3,287 in 2013. KMLE is taken by expected medical school graduates, medical school graduates who failed previous examinations, and those who graduated from overseas institutions. Therefore, the data used in this study are not the findings from several medical schools or estimated data based on sampling but whole data for three consecutive years.
Five characteristics of examinees were set as variables: gender, age, graduation status, written test result (pass or fail), and city of medical school. The gender was divided into male and female; the age was divided into 5 age groups (people in their 20s, 30s, 40s, 50s, and 60s); graduation status was divided into expected graduates and graduates; written test result was divided into a group who passed and a group who failed. With regard to the city of origin, medical schools were divided by location: in South Korea or in other countries. The medical schools in South Korea were divided into two groups: schools in metropolitan cities (including Seoul) and schools in non-metropolitan cities.

Analysis method

A regression model was established, with the score of a written test as a dependent variable and with examinees’ traits as variables. The total score of all variables was analyzed with regression analysis to determine how the variables affected the written test scores in KMLE. For the analysis, IBM SPSS ver. 20.0 (IBM Co., Armonk, NY, USA) was used. The following is the regression model with the total score as a dependent variable and five factors as independent variables.
  • Y=B1X1+B2X2+B3X3+B4X4+B5X5+e

  • Y: Total score

  • X1: Gender, 0=female 1=male

  • X2: Age, 0~4=20’s~60’s

  • X3: Graduation Status, 0=expected graduates 1=graduates

  • X4: Testing Result, 0=failed 1=passed

  • X5: City of Origin, 0~2=overseas~metropolitan cities in South Korea

  • e: Residual

The total score from the 2011 KMLE was 450 points whereas the total scores from the 2012, 2013 examinations were 400 points each. Therefore, the score from the 2011 examination in the descriptive statistics was converted to 400 points. Because the score conversion does not affect the regression analysis, the explanatory power of the regression model or the significance test result is the same as that of the unconverted score. Converted score of the 2011 examination regression co-efficient can be compared with the results of the 2012 and the 2013 examinations.

RESULTS

The present study established a regression model with examinees’ variables and examined the size of the effect and the direction of each individual variable on written test scores. Its original interest was to observe the effect size, based on each variable, but the established model can predict test results. Table 1 showed the descriptive statistics of the total scores of each group, based on the year of KMLE. The explanatory power of the regression model of each year was shown in Table 2. The significance level in the established regression model was 0.05, as shown in Table 3. The explanatory power of each examination was 32% for the 2011 examination, 39% for the 2012, and 38% for the 2013; corresponding variables account for the total score range (range, 32% to 38%). In terms of individual variables, Table 4 showed standardized regression coefficient and non-standardized regression coefficient for each examination number, as well as the significance probability. In all three examinations, the order of significant variables and the effect size of each variable were the same. The regression coefficients in all variables, except the city of medical school, were statistically significant. The variable’s effect in three examinations appeared in the following order: result of written test, graduation status, age, gender, and city of medical school. This means that the scores in controlled conditions differed significantly by gender, age, graduation status, and written test results. However, the score based on city of medical school did not show a significant difference (P > 0.05).
Table 5 compares the regression coefficient of each examination and shows the average difference among the groups. Overall, when other conditions were the same, female students scored 7 points higher than did male students. The younger the examinees were, the higher they scored: people in their 20s scored higher than did people in their 30s; people in their 30s scored 11 points higher than did people in their 40s. On written tests, expected graduates scored 24 points higher than did graduates. With all other conditions the same, examinees who passed the written test are expected to score 62 points higher than those who failed it. The score, based on city of origin, differed in each examination, which implies the effect of examinees who graduated overseas in a particular year. Table 1 showed the number of examinees from overseas: the number is small and the number varies each year; their origins are not equally distributed [1]. In one year, the majority may have come from developed countries; while in another year, all of the overseas examinees may have come from South America or Eastern Europe. Because of differences in groups, the scores were not consistent and cannot be interpreted as a representing a group. The score of overseas examinees had a great effect because there was a very small difference between the examinees from metropolitan cities and those from non-metropolitan cities in South Korea. Therefore, with regard to the scores based on the city of medical school in 2011 and 2012 examinations, domestic examinees scored higher than did overseas examinees; Those from metropolitan cities scored one or two points higher than did those from non-metropolitan cities in South Korea. However, in the 2013 examinations, the overseas examinees scored slightly higher than did the domestic examinees. In this analysis, North Korean defectors were placed in the category of overseas examinees. With regard these examinees, the passing rates varies every year: one person passed the examinations in 2005, one in 2006, one of four in 2007, seven of eight in 2008, three of eight in 2010, and three of seven in 2012 [2].

DISCUSSION

This study analyzed the test scores of three consecutive year KMLEs from 2011 to 2013 and verified the effect of individual variables on test scores. Based on these findings, it is possible to infer with group is most likely to score the highest or the lowest on the written test of the KMLE. One can predict that female medical students in their 20s from metropolitan cities in South Korea are likely to pass the KMLE with a score 63 points higher on average than men in their 50s or over who have graduated. The analysis of the written test scores of the KMLE revealed that female students, younger examinees, and first-time examinees had higher performances. This tendency and the score differences were not limited to one examination, but verified for several recent years. It does not simply suggest that female students will score higher; rather, given the same circumstances, female students are expected to score 7 points higher than male students do. Even in the same gender group, the score can differ by 11 points, based on age. Furthermore, in the same gender and age group, the score can differ by 24 points, based on graduation status. The present study verified that the score difference by a examinees’ city of medical school changes according to the examinees of the year and does not have a general orientation. The location of the domestic medical school also did not affect the performance of the written test, which indicates the standardization of medical school performance in written tests. It focuses on the location of medical schools, such as those in metropolitan versus non-metropolitan cities or domestic schools versus foreign schools, rather than on the differences between individual schools.
It is hard to find the any effects of variables used in this study on nation-based high stakes examination. In one university, it was found that female students have higher academic performances than male students and female students got higher score in KMLE than male [3]. The ratio of female medical students is gradually increasing, and female students are scoring high, even in high-stakes testing [1].Although the data in this study are large-scale data, the data have limitations in that they only show test scores and do not link academic performance before and after the examinations. Furthermore, because of the minimized data collection of examinees’ personal information, the variables that could be used were limited. For follow-up studies, future researchers can analyze whether students display similar academic performance during the school year or analyze test scores based on diverse factors that relate to the cognitive and psychological characteristics of a student, such as specific personal traits, academic performance, or academic motivation.
This study is significant in that the performance differences among examinees on a national level testing could be identified with objective data. When other conditions were controlled, the orientation of the difference and the size of the difference, based on gender or age, were identified. These findings can be used as an index of examinees’ performance levels in KMLE, a foundation for future research on high-performing or low-performing groups, and an aid in the development of teaching methods to improve academic performance.

Notes

No potential conflict of interest relevant to this article was reported.

SUPPLEMENTARY MATERIAL

Audio recording of the abstract.

REFERENCES

1. National Health Personnel Licensing Examination Board. Annual statistic report, 2012 and 2013. Seoul: National Health Personnel Licensing Examination Board;2012, 2013.
2. Kim YH. North Korean defectors seeking health certification to take the National Medical Licensing Examination in the Republic of Korea: figures and procedures. J Educ Eval Health Prof. 2012; 9:121–123. http://dx.doi.org/10.3352/jeehp.2012.9.12.
crossref
3. Ahn D, Im H, Kim KS. Correlation study of the scores of entrance examination, graduation examination and National Licensure Examination at Korea University Medical College. Korean J Med Educ. 2000; 12:91–96.
crossref

Table 1.
Descriptive statistics of the total scores of each classified group of test takers of Korean Medical Licensing Examination from 2011 to 2013
Exam year Variable Subcategory Mean Standardized deviation Minimum value Maximum value Number of examinees
2011 Gender Female 307.6 27.2 185.8 368.4 1,077
Male 297.0 32.0 32.0 371.6 2,287
Age People in 20s 305.1 27.4 32.0 371.6 2,621
People in 30s 285.8 35.2 143.1 362.7 714
People in 40s 233.6 33.3 163.1 298.7 25
People in 50s 236.9 14.9 221.3 251.1 3
People in 60s 212.9 - 212.9 212.9 1
Graduation status Expected graduates 303.4 28.2 32.0 371.6 3,159
Graduates 253.9 34.2 143.1 321.3 205
Testing result Failed 244.2 43.4 32.0 342.7 215
Passed 304.2 25.7 234.7 371.6 3,149
City of origin Overseas 254.9 10.9 238.2 269.8 7
Non-metropolitan cities in South Korea 298.7 30.9 54.2 368.4 1,088
Metropolitan cities in South Korea 301.3 30.8 32.0 371.6 2,269
Total 300.4 30.9 32.0 371.6 3,364
2012 Gender Female 302.0 25.8 197.0 359.5 1,244
Male 290.1 32.7 100.5 372.5 1,933
Age People in 20s 300.5 27.1 187.5 372.5 2,128
People in 30s 284.7 33.0 100.5 353.5 1,012
People in 40s 247.4 45.8 148.5 318.5 33
People in 50s 221.6 30.5 181.5 247.0 4
Graduation status Expected graduates 297.4 27.1 160.0 363.5 3,040
Graduates 237.7 47.5 100.5 372.5 137
Testing result Failed 207.9 25.5 100.5 233.5 120
Passed 298.2 25.5 234.0 372.5 3,057
City of origin Overseas 218.0 84.9 100.5 330.0 7
Non-metropolitan cities in South Korea 294.3 30.6 146.0 372.5 1,056
Metropolitan cities in South Korea 295.3 30.2 142.5 363.5 2,114
Total 294.8 30.7 100.5 372.5 3,177
2013 Gender Female 307.4 26.4 148.0 375.0 1,313
Male 297.1 30.7 123.0 364.0 1,974
Age People in 20s 306.8 25.9 186.0 375.0 2,256
People in 30s 290.8 30.3 123.0 363.0 988
People in 40s 247.8 50.7 147.0 335.0 38
People in 50s 210.8 41.7 148.0 258.0 5
Graduation status Expected graduates 303.9 25.6 186.0 375.0 3,132
Graduates 245.1 43.0 123.0 349.0 155
Testing result Failed 213.1 23.5 123.0 239.0 110
Passed 304.2 24.5 240.0 375.0 3,177
City of origin Overseas 247.9 60.5 148.0 349.0 20
Non-metropolitan cities in South Korea 301.6 27.5 168.0 375.0 1,229
Metropolitan cities in South Korea 301.5 29.7 123.0 373.0 2,038
Total 301.2 29.5 123.0 375.0 3,287
Table 2.
Explanatory power of the regression model of each year for Korean Medical Licensing Examination from 2011 to 2013
Exam year R R2 Adjusted R2 Standard error
2011 0.567 0.321 0.320 25.4793
2012 0.620 0.385 0.384 24.1143
2013 0.616 0.380 0.379 23.2200
Table 3.
Significance probability in the established regression model by analysis of variance
Exam year Model Sum of square degree of freedom Mean square F P-value
2011 Regression 1,032,344.545 5 206,468.909 318.038 0.000
Residual 2,179,998.418 3,358 649.195
Total 3,212,342.963 3,363
2012 Regression 1,154,209.424 5 230,841.885 396.977 0.000
Residual 1,843,933.984 3,171 581.499
Total 2,998,143.408 3,176
2013 Regression 1,083,334.248 5 216,666.850 401.869 0.000
Residual 1,768,943.634 3,281 539.148
Total 2,852,277.883 3,286
Table 4.
Standardized regression coefficient and unstandardized regression coefficient for each examination year, as well as the significance probability for Korean Medical Licensing Examination from 2011 to 2013
Exam year Model unstandardized coefficient, B Standard error Standardized regression coefficient, Beta t-value P-value
2011 (Constant) 287.676 3.782 76.069 0.000
Gender -7.017 0.950 -0.106 -7.390 0.000
Age -11.224 1.044 -0.163 -10.753 0.000
Graduation status -27.633 1.991 -0.214 -13.876 0.000
Result of written test 45.367 1.922 0.359 23.602 0.000
City of origin 1.025 0.936 0.016 1.094 0.274
2012 (Constant) 257.402 3.849 66.879 0.000
Gender -7.948 0.883 -0.126 -9.001 0.000
Age -11.444 0.866 -0.188 -13.213 0.000
Graduation status -23.734 2.389 -0.157 -9.937 0.000
Result of written test 70.231 2.584 0.436 27.184 0.000
City of origin 1.490 0.897 0.023 1.660 0.097
2013 (Constant) 262.577 3.719 70.608 0.000
Gender -7.290 0.834 -0.121 -8.742 0.000
Age -10.455 0.846 -0.178 -12.355 0.000
Graduation status -20.132 2.264 -0.145 -8.891 0.000
Result of written test 70.761 2.621 0.432 26.996 0.000
City of origin -0.079 0.818 -0.001 -0.097 0.923
Table 5.
Difference of regression coefficient of each examination by group and the mean difference among the groups for Korean Medical Licensing Examination from 2011 to 2013
Factor Result 2011 2012 2013 Mean
Gender Female>male -7.017 -7.948 -7.290 -7.418
Age People in 20s>people in 30s>people in 40s>people in 50s>people in 60s -11.224 -11.444 -10.455 -11.041
Graduation status Expected graduates>graduates -27.633 -23.734 -20.132 -23.833
Result of written test People who passed>people who failed 45.367 70.231 70.761 62.120
City of origin Metropolitan cities in South Korea>non-metropolitan cities in South Korea>overseas 1.025 1.490 -.079 0.812
TOOLS
Similar articles