PDF
Citation
Print
INTRODUCTION
Currently, the dementia population and management costs are expected to increase along with the increase in the elderly population worldwide. The National Statistical Office reported that the elderly, aged 65 years or older, accounted for 15.7% of the South Korean population in 2020 and predicted that they would account for 20.3% in 2025 and 46.5% in 2067 [23,24]. Considering that the elderly population constitutes 9.1% of the world’s population, the increase in the elderly population in Korea is high [25]. Dementia is a typical mental illness that is closely related to the increase in the elderly population; its main symptom is cognitive impairment, including memory degradation and degradation of daily life functions because of changes in personality [1]. Given that the prevalence of dementia in the Asian population is 5.23% [5], the estimated prevalence of dementia above 65 years of age, which reaches 10.53%, and the annual management cost per dementia patient, which is up to KRW 18.51 million, shows the risks for South Korean elderly [9,17]. Aside from dementia, the number of patients is also expected to increase as the elderly population increases; consequently, the need for research and development related to dementia has increased.
In Korea, various support systems, such as community dementia reassurance centers, long-term care services, and medical expenses, are implemented through the national responsibility system for dementia and state affairs. Specifically, the government continues to develop policies that cover the costs of diagnosis and treatment of patients with dementia, such as the application of health insurance for dementia screening costs and introduction of a special calculation system for severe dementia [16]. However, dementia, a degenerative disease, worsens with time, and its risk increases with age; therefore, the social cost must be limited by preventing or alleviating the occurrence and deterioration of dementia. Efficient solutions to dementia-related problems, such as early detection and quick intervention, are required. Moreover, considering the persistent COVID-19 situation, it is expected that the need for non-face-to-face psychological tests will increase rapidly. Nonface-to-face neuropsychological testing is useful in that it can overcome the limitations of time and space required for conducting psychological test. Based on smart devices or online platforms, the psychological characteristics of examinees can be captured using the paradigm of neuropsychological tests without direct contact between examiner and examinee. In particular, the fact that dementia patients, which are one of the main subjects of neuropsychological testing, are often difficult to hang out alone, suggests the need for non-face-to-face neuropsychological testing. In this respect, Korean Repeatable Battery for the Assessment of Neuropsychological Status (KRBANS) as a screening neuropsychological test is not only more sophisticated and systematic than tests such as MiniMental State Examination (MMSE) and clinical dementia rating (CDR) scale, but also has the advantage of conducting test shorter and more efficient than diagnostic tests such as Seoul Neuropsychological Screening Battery (SNSB)-II.
In studies and markets related to dementia, it is expanding around diagnosis in South Korea. In particular, the demand for dementia testing tools that can help predict high-risk dementia groups and prevent dementia is increasing. Screening tests for identifying dementia in Korea include the Korean version of Consortium to Establish a Registry for Alzheimer’s Disease (CERAD-K) [18] and SNSB [15]. These screening tools, used in approximately 40 countries, are comprehensive and inclusive tools that take approximately 30 to 40 minutes for CERAD-K, and approximately 1 hour and 30 minutes to 2 hours for SNSB. However, these screening tests require space and time for trained professionals, resulting in inconveniences and limitations in access and usefulness to prevent dementia. In addition, relatively brief and easy tests developed to identify dementia or mild cognitive impairment (MCI), such as MMSE [10] and CDR [19], are commonly used in clinical settings, but limitations in their accuracy and clinical usefulness have been highlighted [11,26]. To compensate for these problems, there is a need for a tool that can be used to identify and prevent dementia with accuracy and clinical usefulness while reducing the restrictions on places and efforts needed for testing as much as possible. In this context, the K-RBANS is drawing attention.
The RBANS, which consists of 12 subtests and takes approximately 20 to 30 minutes when conducted by an experienced examiner, is an individual simple test that measures attention, language, visual space composition ability, immediate memory, and delayed memory [22]. The K-RBANS [21], standardized for the Korean population, was conducted with the examiner using test papers, targeting individuals between 12 to 60 years or older. Standard scores were collected from 606 healthy adolescents and adults using stratified sampling. In particular, the K-RBANS was developed for three purposes. First, as an independent core battery, the K-RBANS was developed to identify and evaluate senile dementia. Second, as a screening battery, the K-RBANS was considered to assess patients who are unable to or unsuitable for standardized assessments, consuming relatively long time and effort. Finally, it was developed in a format that allows repeated evaluations when homomorphism is required to control the practice effect. Although the K-RBANS is designed to be conducted as easily as it is on the bed, there are still inconveniences in the administration and scoring processes. Therefore, this study minimizes inconvenience by materializing the K-RBANS based on tablet PCs and confirms the effectiveness of the tablet-administered K-RBANS using Clinical Data Interchange Standards Consortium [3] that enables non-pharmacologic dementia prevention and early detection by examining equivalence with the existing K-RBANS.
MATERIALS AND METHODS
The study protocol was approved by the Institutional Review Board of Yeungnam University Hospital (IRB No. YUMC 2019-06-054).
Participants
In this study, we recruited 88 participants from a university hospital. A neuropsychological evaluation was conducted for patients with dementia or patients with suspected dementia who voluntarily agreed to participate in the study, from September to October 2019. The inclusion criteria were a Korean Mini-Mental Status Examination (K-MMSE) score between 10 to 26 and a CDR score between 0.5 to 2. The exclusion criteria were those patients who 1) required frequent admission or hospitalization, 2) required medical re-access due to rapid deterioration of dementia symptoms, or 3) used internal medicine or had accompanying diseases that may cause cognitive decline.
Materials
The K-RBANS-update
The K-RBANS [21] is an independent and concise examination of attention, language ability, visuospatial/constructional ability, and immediate and delayed memory. This test consists of 12 subtests and is designed to allow skilled examiners to conduct the test for approximately 20 to 30 minutes. Immediate memory consists of list learning and story memory, which measures the examinee’s ability to remember immediately after the stimulus is presented. Visuospatial/construction capabilities are indicators consisting of figure copy and line orientation, indicating the ability to copy visual simulations accurately and perceive the spatial relationships of pictures. Language ability consists of picture naming and semantic fluency, reflecting the examinee’s ability to verbally recall or name learned objects. Attention refers to the ability to remember and manipulate visually and linguistically presented information in a short period, which is measured by digit span and coding. Delayed memory evaluates the ability of proactive memory, including word list recall, recognition, story, and figure recall. Participants’ overall cognition ability could be reflected by the total scale index (TSI), which integrates the above five index scores. In addition, the K-RBANS was designed to target individuals between 12 to 60 years or older, and the norm score was collected and standardized from 606 healthy adolescents and adults using stratified sampling. Moreover, the K-RBANS, developed as a tablet version, was created by Clupea® (Clupea, Daegu, Korea) as an application in accordance with the instructions and procedures of the test paper and manual. Visual stimuli were presented in the tablet according to what was presented in the test paper and stimulus booklet, and instructions and auditory stimuli were presented in a recorded voice. In this process, mental health professionals and two clinical psychologists ensured that the KRBANS stimuli and instructions were properly presented. In this study, we implemented the tablet version K-RBANS on a 10.5 inch screen using Samsung Galaxy Tab S5 (Samsung, Seoul, Korea).
The K-MMSE
The K-MMSE is an assessment tool developed by Folstein et al. [10] for overall cognitive capabilities and cognitive adverse functions. In Korea, Kang et al. [14] proposed the K-MMSE, which is widely used in clinical evaluations and research on patients with dementia. The K-MMSE is composed of 30 points : time orientation (5 points), spatial orientation (5 points), memory registration (3 points), attention and calculation ability (5 points), memory recall (3 points), language ability (8 points), and drawing ability (1 point). Although the MMSE cut-off scores differ depending on the purpose of the study, it is known that scores between 24 and 28 are mainly used to differentiate between patients with MCI and dementia [13,14,20]. In this study, in terms of practicality in the actual clinical setting, the MMSE score of 26 and the CDR score of 0.5 were selected according to the domestic medical care benefit application standards.
CDR scale
The CDR scale [19] is a semi-structured interview with patients and caregivers. This scale evaluates six cognitive functional areas : memory, orientation, judgment and problemsolving, social activities, household activities and hobbies, and hygiene management, scoring 0 for undamaged; 0.5, suspected to damage; and 1 to 5, mild, moderate, severe, and very severe cognitive impairment, respectively. The scores evaluated in all six areas are calculated to “sum of boxes,” ranging from 0 to 30 points. Moreover, the overall CDR score is determined using memory as a key factor, whether high or low, among the five cognitive domains, or following the prescribed algorithm.
Procedure
Patients who voluntarily agreed to participate received a consent form to participate in the study, conducted a simple physical examination, and collected the test results within a year from the date of participation. Participants who did not undergo the K-MMSE or CDR were tested on the day they visited and were registered to participate in the study and assigned to the odd or even groups. Each group was administered with the K-RBANS according to a set order of subtests that alternated between paper and tablet formats. The order of administration of the K-RBANS between the groups is presented in Table 1. This procedure enables the estimation of scaled scores for the tablet-administered K-RBANS using norm samples and compares the format effect between paper-and tablet-administered K-RBANS. If a substitution is needed because of a dropout, the number assigned to drop-out participants is given in the order of screening numbers. The test was conducted in a room where noise was blocked, with no distractions, and the K-RBANS was performed in an order by preparing devices for the tester and participants. Compensation for participating in this study was provided to those who completed the test.
Design
To test equivalence, we used a non-randomly equivalent group design [8]. A non-randomly equivalent group design can be used if two or more subtests exist to measure one ability construct, and if the norm sample is constructed through appropriate standardization procedures. The performance of the norm sample obtained through the standardization process is used to estimate and compare performance using the tablet version and finally compare the media effect (difference between the existing version and tablet version). Participants in this study were divided into the odd and even groups according to the test subject number (RXXX), and each group was examined in the order shown in Table 1. In this design, the K-RBANS norm samples as data that are conducted in a papertype format were utilized to derive a formula for predicting conversion scores to determine the media effect of the KRBANS tablet version. In this study, we compared the differences in the raw scores of the K-RBANS administered on paper and tablets and conducted a regression analysis based on age, number of years of education, gender, and covariate subtests to confirm the media effects of the K-RBANS. Therefore, half of the subtests served as paper-administered covariate tests because the norm and study samples were both administered in paper format, while the other half of the subtests were administered in tablet format to evaluate the format effect. This approach was adopted to maximize the multiple correlations of each subtest in both the halves, which maximizes statistical power [8]. In addition, average residuals derived from the difference between observed and predicted scaled scores on tablet-administered subtests imply that the scores from the tablet PC format were higher than predicted in positive value, and vice versa. The effect size was calculated by dividing the average residuals by 3, which is the standard deviation of the scaled scores in the norm sample.
Statistical analysis
Prior to the analysis, we excluded cases in which the test results were deemed contaminated because of failure to complete the test or indirect participation in the test. We compared demographic information between the odd and even groups and raw scores of the K-RBANS according to format using an independent t-test (Fisher’s exact test for gender distribution by groups). Next, we used multiple regression analysis to estimate scaled scores of the tablet-administered K-RBANS based on the effects of sex, age, and education and analyzed format effects using a one-sample t-test of average residuals between observed and predicted scaled scores. In addition, the effect size was calculated to determine the extent of the difference between administration types to confirm format effects rather than judging on dichotomous significance. We also verified the capacity of the tablet-administered KRBANS to screen for cognitive impairment using a receiver operating curve (ROC) analysis. The assessment of statistical significance in all analyses was set at p<0.05. The data were analyzed using IBM SPSS Statistics for Windows, version 25.0 (IBM Corp., Armonk, NY, USA) [12].
RESULTS
Demographic information
Of the 88 participants, the average age in the odd and even groups was 67.75±8.75 and 68.43±7.93, respectively. The odd group consisted of 12 men (27.27%) and 32 women (72.73%), and the even group consisted of nine men (20.45%) and 35 women (79.55%). The average education years in the odd and even groups were 9.55±4.82 and 8.32±4.36, respectively. In the screening phase, the odd group had an average K-MMSE score of 24.25±2.67 points and a CDR score of 0.69±0.38, while the even group had a K-MMSE score of 24.45±2.35 and a CDR score of 0.63±0.22. The demographic information of the study participants in terms of age, gender, education, and cognitive ability did not show significant differences between the odd and even groups. The results are presented in Table 2.
Comparison of raw scores between original and tablet versions of the K-RBANS
In Table 3, a comparison of the original K-RBANS conducted in paper and tablet formats showed significant differences in the type of performance in line orientation (t=-2.94, p<0.001). In addition, the effect size between the type of K-RBANS performance was high in line orientation (d=0.86). However, it showed moderate effect sizes in picture naming (d=0.34), semantic fluency (d=0.42), symbol coding (d=0.34), and word list recall (d=0.35) while other subtests showed low effect sizes.
Estimating the scaled scores of the tablet version of the K-RBANS using norm samples
In this study, we utilized data from a normative group to estimate the scaled scores of the tablet version of the K-RBANS. In Table 4, different formulas were estimated using regression analysis based on the following age groups : 40 to 49 years, 50 to 59 years, and 60 years or older. Table 5 shows how well the predictive formulas derived from the norm sample fit the tablet version of the K-RBANS. The correlation coefficients of the norm sample represent the results calculated for each subtest using the prediction formula. All subtests showed correlation coefficients ranging from 0.99 to 1.00. In the tablet version of the K-RBANS, the correlation coefficient ranged from 0.85 to 1.00, indicating that the formula calculated from the normative group was also useful in the tablet version.
Media effects of the tablet version of the K-RBANS using estimated scaled scores
Table 6 shows the analysis of the media effects using observed and predicted scaled scores with tablets, not using scaled scores of the paper-type subtest. The analysis of the difference between the observed and predicted scaled scores (format effects) with a one-sample t-test showed significant differences in the case of zero residual in all subtests of picture naming and in all delayed memory subtests. The predicted scaled score of picture naming showed positive average residuals, while all subtests of delayed memory showed negative average residuals. In addition, the effect size of word list and story recall showed a moderate level in delayed memory, and other subtests showed a low effect size that did not exceed 0.2.
ROC analysis results
To verify the capacity of the tablet version of the K-RBANS to detect performance impairment in patients with dementia and MCI, we performed ROC analysis using estimated scaled scores in paper- and tablet-administered K-RBANS. The area under curve (AUC) of the TSI was the highest at 0.827, with a cut-off point of 67.5. In the other five indices, the AUC was shown to be statistically significant, ranging from 688 to 820. In addition, the sensitivity of the TSI was 80%, and the specificity was 76%. The results are listed in Table 7.
DISCUSSION
Recently, there has been a movement to establish a psychological examination paradigm using smart devices and PCs. Q-interactive is an iPad-based examination system that allows clinical evaluations, such as the Wechsler Intelligence Scale for Children-fifth edition, Wechsler Individual Achievement Test-third edition, and Clinical Evaluation of Language Fundamentals-fifth edition, to be conducted, scored, and reported [27]. In Korea, CERAD-K manufactures and distributes applications that can be used in smart devices. However, despite having technology that can implement the neuropsychological examination paradigm in smart devices in Korea, there is not enough verification of its effectiveness. For example, in the case of CERAD-K, the equivalence and effectiveness of applications for smart devices have not been verified, which limits the interpretation of the test results. In this study, we aimed to confirm whether the K-RBANS performance differs in paper and tablet administrations and determine whether the tablet-administered K-RBANS can detect and identify patients with dementia and MCI.
We compared paper- and tablet-administered K-RBANS performances using raw scores. In the results of the one-sample t-test, line orientation showed a significantly low score in the tablet-administered K-RBANS. Moreover, line orientation showed a large effect size; picture naming, semantic fluency, coding, and word list recall showed medium effect sizes; and other subtests showed low effect sizes. Given that raw scores of the tablet-administered K-RBANS have limitations using the norm sample of paper-administered K-RBANS because it would be easily affected by age, education, and gender, raw scores would be more sensitive to the format effect. However, it needs to be refined carefully so that the equivalence of raw scores can reduce potential threats to the adaptation of psychological tests [6].
Next, we estimated scaled scores of the tablet-administered K-RBANS to verify format effects using demographic variables and covariate subtests with the norm sample. In this process, we produced prediction equations derived from scaled scores of norm data and estimated scaled scores in the tablet-administered K-RBANS to confirm the format effect. Consequently, in a one-sample t-test, picture naming, word list recall, list recognition, story, and figure recall showed significant differences but low effect sizes, and word list and story recall showed low effect sizes. This result means that some subtests differ when the performance of the tablet-administered K-RBANS is converted into a scaled score of the norm sample. Specifically, with respect to the predicted scaled scores with high correlations to the norm sample, observed scores that are smaller than predicted scores in the tablet-administered K-RBANS suggest that participants’ performance in word list and story recall partially meet the expected level. However, given that some psychological tests that were administered digitally showed format effects at this level [2,4,7,8], the results that showed low format effects of the tablet-administered K-RBANS appeared to be acceptable. In particular, the average residuals of delayed memory showed a negative value, indicating that the predicted scaled scores were higher than the observed scaled scores. This result suggests that the scaled score is a proper measurement for the soothing effects of demographic variables such as age and that future studies need to elaborate this further, because actual performance was lower than predicted in delayed memory. In addition, in the signal detection analysis for the tablet-administered K-RBANS, all indices, including the TSI, were found to be significant, and the sensitivity and specificity of the TSI were the most suitable for detecting cognitive impairment.
However, this study has several limitations. First, there was no control for matching gender, age, and education between the groups. Although we used scaled scores of the tablet-administered K-RBANS based on age and education, it is known that neuropsychological assessment could be affected by demographic variables. We had a limitations in comparing performance between paper- and tablet-administered K-RBANS based on age, gender, and education directly because of the number of participants and their distribution. Second, we did not measure all the performance on the tablet-administered K-RBANS; therefore, it was not possible to compare test results between paper and tablet administrations directly within participants’ capacity. If possible, future research should confirm the test-retest reliability.
Nonetheless, the study results suggest that the K-RBANS has adherent reliability and validity for neuropsychological assessment tools in paper administration and can also be used in tablet administration for screening cognitive decline or impairment because of dementia. This could make users relatively free of limitations related to testing time and place, by administering neuropsychological tests in electronic devices such as tablet PCs. The conversion of K-RBANS to a smart device can contribute to early detection of subjects with mild neurocognitive impairment or dementia. It is thought to be useful in cases where it is difficult to move alone to receive a neuropsychological test due to reduced function or inconvenience in movement. Furthermore, the conversion of neuropsychological testing to smart devices can be a stepping stone to overcome the temporal and spatial limitations of traditional neuropsychological testing when linked with an online platform.
CONCLUSION
In this study, we confirmed equivalence of tablet-administrated K-RBANS using non-randomly equivalence group design. First, we found that raw scores had some limits for scaled scores using norm samples that was collected by standardization of paper-administrated K-RBANS. Second, in order to estimate the equivalence of tablet-administrated K-RBANS performance correcting effects of demographic characteristics, the formula were yielded from the norm sample. Using this formula, we compared observed and predicted scores, and verified equivalence of K-RBANS enough to use norm sample. Finally, tablet-administrated K-RBANS, as a screening tool, showed good sensitivity and specificity.
XML Download