Doctors are required to continuously update and improve their medical skills and knowledge based on changes in the field that lead to better practices in medicine, and to voluntarily learn what is necessary to meet the requirements for medical professionals [1,2]. Therefore, individuals have to decide their own learning needs to plan and implement these processes for their successful lifelong professional development [1]. From this perspective, self-directed learning has always been a cornerstone of the ideal student learning methods [3].

Students need to be able to accurately evaluate their competency of curricular topics and modulate learning goals accordingly to be successful self-directed learners [4]. Formative assessment is very important as an instructional tool in this regard [5], because it can be considered not only as an assessment resource but also a guide for students to recognize areas where they experience difficulties in their acquisition process by tracing their own progress in self-directed learning [6]. In general, the term “formative assessment” includes any activities that happen between trainers and trainees after an assessment [7]. Therefore, formative assessments are designed to help students improve learning by providing summative assessment familiarization and feedback that guides student learning [8].

Although there are various kinds of formative assessment tools, quizzes are used most often [9-11]. This is because they promote motivational completion by increasing medical student and teacher interactions [12]. Repeated testing enhances long-term information retention compared to repeated studies [13]. This implies that testing is not only an assessment tool, but also plays a significant role in student learning. Furthermore, information retrieval demonstrated by taking tests is a key to effective long-term information retention [14]. As a result, when quizzes are implemented as a method of test-enhanced learning, they can be useful for students’ learning complex sets of medical facts [15].

Quizzes where students are expected to strengthen their learning by completing the quiz activity are called formative quizzes. Formative quizzes have been reported to improve summative assessment outcomes [16-18]. Although many studies have reported the usefulness of quizzes for improving learning achievements, they have not evaluated various quiz activity conditions with respect to the quiz’s relationship to the summative assessment outcome. In other words, critical points within the quizzes should be included when determining positive learning effects. Currently, only quiz scores or quiz activity completion are used when analyzing the relationship between the quizzes and summative assessment outcome.

Much information about the relationship between formative assessments and summative assessments has already been reported in previous studies. However, the association between various quiz activities such as quiz activity frequency and sequential quiz activity with learning accomplishments has not been sufficiently validated. Therefore, we evaluated the effects of various analytical quiz activities on summative assessment outcomes in this study [19]. Several critical quiz activity points, such as quiz activity frequency and score trends for sequential quiz activities, were analyzed to determine the factors that played a role in improving summative assessment outcomes. We attempted to explore how quiz activity affected summative assessment outcomes by analyzing the data from 3 consecutive years to compare the average and final quiz scores, quiz activity frequency, and summative assessment outcomes.

A total of 217 students’ data were obtained for 3 consecutive years and used for analyses. The number of students enrolled for this study in 2019, 2020, and 2021 were 74, 72, and 71, respectively.

Formative quizzes were implemented for self-directed learning and formative assessment, which are usually expected to accompany feedback from trainers. The purpose of formative quizzes in this study was not the same as those in traditional learning environments as described in the methods section. The formative quizzes were devised for students to recognize what they needed to improve upon for further studying. This was the main intention of the formative quizzes. It is assumed that any students who participated in supplementary study to compensate for a lack in knowledge during the formative quiz activities could obtain better scores on summative assessments.

Given that students were informed during the first-class period of the course that the formative quiz scores would not be included in their final grades, it is believed that the formative quizzes were mainly utilized by students as a measuring tool for their status of learning. We recognize the possibility that there were some students who had previous knowledge of the content before answering the formative quizzes. However, the formative quiz could still guide students’ learning via formative questions regardless of any prior exposure to the quiz content.

There was a significant summative assessment outcome difference between students who completed their quizzes and those who did not. This is concordant with previous studies [17,18]. The summative assessment measures the extent of learning while the formative assessments are a tool to help guide students toward their learning goals. There is a plethora of evidence that formative assessments are associated with positive learning outcomes [6,21,22]. Feedback during formative assessments was assumed to be a core component for positive summative assessment outcomes [21]. Medical students want feedback in a timely manner, either verbally, aurally, through video, or via self-assessment [23-26]. Feedback should be different depending on the information or skills the student needs. With the intent to enable self-directed learning, we focused on students using quizzes to identify their knowledge gaps. The quiz questions focused on key medical informatics concepts and knowledge.

In general, formative assessments are low stress and students do not feel threatened and judged when taking them [27]. The formative quizzes did not have a completion time and scores were not included in the final grades. Since the quizzes were not mandatory, it is likely that they did not overwhelm the students; it is plausible that the students who completed them were more motivated to learn the material than those who did not since there were no other included incentives regarding their participation [28]. We assumed the students who completed the quizzes were more apt to voluntarily and vigorously reflect upon their learning goals during the formative assessment portions of the course.

Repeated exposure to testing enhances self-efficacy on tests [29]. Information provided repeatedly over time is more easily retained than when all information is offered at once [30]. Information retention is more enhanced on delayed tests rather than repeated studying [31]. As the quiz frequency increased, the summative assessment scores increased (Table 1). Although the summative assessment outcome differences in each group were not statistically significant and the number of students in the high frequency groups was not high enough for statistical analysis, we still observed an increasing trend in the summative assessment score. The time intervals between quizzes spanned up to 100 days (data not shown). We created several combinations of quiz frequency groups to reinforce analytical power related to the low student numbers in high frequency groups. Although not statistically significant, the summative outcome scores increased as the quiz frequency combinations increased to the highest frequency combination group (group 3) (p=0.075 and p=0.067 in t-test and Mann-Whitney test, respectively) (Table 2). This increase might be caused by the repeated testing over time, which likely facilitated information retention and helped students better prepare for subsequent testing.

Taking quizzes repeatedly was self-directed because there was no other participation incentive. When incentives were introduced, the number of students who scored well on the quizzes did not correspond to the number of students who scored well on the summative assessment in the previous study [10]. This result may be explained by the assumption that self-directed learning is strengthened when it is implemented voluntarily, without any external pressure.

As repeated tests have shown over time, we assumed that the final quiz score would correspond to the summative assessment outcome. Interestingly, there was no statistically significant correlation between the final quiz score and the summative assessment outcome (Fig. 3). This strongly implies that the formative quiz scores are not a direct predictor of summative assessment outcomes. Instead, the main role of formative quizzes is not just to provide knowledge, but to also enable students to understand in which areas improvement was needed. Although positive correlations between quiz scores and summative assessment outcomes have been reported in previous studies [32-36], the educational conditions in other studies were not fully comparable to those of the current one, which could explain the differences in the correlational results. In our study, students were not concerned about quiz scores because they were not included in the final grades nor did the students feel judged by their tutors, two factors that appeared to act positively on the students’ self-directed learning.

This study had some limitations. First, it focused specifically on the quiz activity’s relationship to the summative assessment outcome. It did not consider other factors such as individual tutor feedback or other available resources. Second, students took the quizzes voluntarily, which does not always mean that students who abstained from taking the quizzes did not participate in another form of self-directed learning. We didn’t analyze other possible self-directed learning actions unrelated to formative quizzes. Third, since this study was retrospectively analyzed with data recorded during a medical informatics course over 3 consecutive years, it does not include any information asking for direct responses of students regarding the degree of compliance to the intention of the formative quizzes. Fourth, we showed a significant difference in summative assessment outcomes between the quiz completion and quiz incompletion groups. This result could be reinforced through the further analysis of the difference in summative assessments depending on the students’ academic performances of all learning activities. It could not be determined whether the significant difference of summative assessments between the two groups was caused solely by the quiz activity or if it was affected by the excellence in the students’ learning abilities in total learning activities.

In conclusion, self-directed learning using quizzes is thought to be useful for a better summative assessment outcome regardless of frequency and the final score obtained. Students who performed better on their summative assessments are assumed to have improved in their weaker areas through the quiz learning activities. From this perspective, students used the quiz learning activities to overcome their lack of knowledge. The quizzes themselves did not provide direct knowledge, but instead revealed their weaker points, and were believed to have motivated them to voluntarily make up for those insufficiencies. More devices for self-directed learning need to be developed and recommended to students to help them voluntarily improve their performance. Furthermore, it is suggested that additional studies be conducted in order to analyze the difference in the summative assessment outcome according to the level of students’ excellence in their academic performance.