INTRODUCTION
Although computerized adaptive testing (CAT) has been widely used for large scale examinations and for the assessment of the psychological indices, no freely available, well-documented web-based program is known. An ability estimation algorithm was first described by Baker [1] that can be used for a personal computer base. Kim et al [2] at Busan University in Korea published the java source for a CAT in a textbook first in Korean. However, nowadays the user interface for examination has rapidly changed from personal computer-based to web-based. For a CAT system, the following four components are necessary:
There is no integrated program freely available to execute CAT in the educational fields including the above four categories. Therefore, to provide an open access web-based computerized adaptive testing (CAT) program based on the Rasch model and 2- and 3-parametric logistic models, we tried to make a web-based ability estimation program after constructing the management system of examination, including an item bank.
MATERIALS AND METHODS
Tools used for programming were as follows:
Algorithm
The key algorithm was already known for the ability estimation [2]. It was coded for web-inferface.
RESULTS
Input of items
Items can be inputted on a website. At first, basic information, such as category, item type, key words, IRT model, item parameters already determined, answer option form and number of answer option (Fig. 1), are added. An item stem can be inputted for text, and flash files can be inputted for multimedia data such as images, sounds and movie (Fig. 2). Items are presented randomly to each examinee. The table structures are shown in Fig. 3.
Search within items
On the screen, key words or text words from the items can be used to search. A Boolean search with two categories is also possible (Fig. 4). As for the CAT, each item that was already analyzed according to the respective IRT model can be searched. Each item’s information function is viewed, and the rate of options by examinees in the CAT is also presented (Fig. 5).
Creation of test and the addition of items
To create the test, the title, start and stop time, and type of test should be inputted (Fig. 6). Available types are computer-based testing, Rasch model, and 2- or 3-parametric logistic models. The addition of items to the test can be done by selecting items in the item bank or by adding new items manually. Added items are automatically registered in the item bank.
Estimation of the examinee’s ability
Estimation of the examinee’s ability was done according to the maximum likelihood estimation for the Rasch model and 2- or 3-parametric logistic models. The theoretical process was referenced from Kim et al2. Unlike Kim et al’s source codes, the present source codes can estimate the ability of examinees from multiple examinations because this present one provides a common structure for item information functions, regardless of the number of examinations. Kim et al’s source codes are only applicable to one examination. A flow chart of the ability estimation was presented in Fig. 7. Source codes for the estimation of an examinee’s ability are presented in Appendix 1.
Viewing the results of an examination
After the termination of an examination, the results can be shown immediately. The examinee’s ability is shown (Fig. 8). Each examinee’s response to each item can be seen after clicking on an examinee’s name (Fig. 9). Since the results of the examination were analyzed on the web, there is no need to download result files and analyze them again.
CAT program
Source code of CAT in this paper is freely available at https://sourceforge.net/projects/irt-cat/
Execution of the CAT in the field
The CAT program in this paper has already been implemented in the educational field with easy interpretations of examinees’ ability [3].
DISCUSSION
The merits of the present program for the estimation of an examinee’s ability are that it is web-based and allows for the immediate estimation of ability after the termination of testing. Also, the selection of the less exposed items was prompt because the results of exposure are recorded in the system. Practical issues in developing and maintaining a CAT program included an item bank, test administration, test security and examinee issues [4]. Out of these, item bank and test administration was also actually addressed in the IRT-CAT program. Test security and examinee issues should be dealt according to the policy of each institute during the implementation of the CAT program. Since it is difficult to get a freely available CAT program, IRT-CAT may open the horizon for an easily applicable web-based CAT. It may be able to be used widely for large-scale examinations, class-based examinations, equating studies among many educational institutes, and psychological examination in the clinic. This system may still not be complete. Therefore, it will be helpful to other researchers to transform the program for their own purposes taking advantage of this program’s open source code. We would like to add the system of parametric estimation for pretest items that are not used for the ability estimation according to item response theory after termination of examination to present IRT-CAT program.