Journal List > J Clin Neurol > v.9(2) > 1048429

Kim, Jeong, Han, Ryu, Lee, Ryu, Lee, Shim, and Choi: Reliability and Validity of the Short Form of the Literacy-Independent Cognitive Assessment in the Elderly

Abstract

Background and Purpose

The Literacy-Independent Cognitive Assessment (LICA) has been developed for a diagnosis of dementia and is a useful neuropsychological test battery for illiterate populations as well as literate populations. The objective of this study was to develop the short form of the LICA (S-LICA) and to evaluate the reliability and validity of the S-LICA.

Methods

The subtests of the S-LICA were selected based on the factor analysis and validation study results of the LICA. Patients with dementia (n=101) and normal elderly controls (n=185) participated in this study.

Results

Cronbach's coefficient alpha of the S-LICA was 0.92 for illiterate subjects and 0.94 for literate subjects, and the item-total correlation ranged from 0.63 to 0.81 (p<.01).The test-retest reliability of the S-LICA total score was high (r=0.94, p<.001), and the subtests had high test-retest reliabilities (r=0.68-0.87, p<.01). The correlation between the K-MMSE and S-LICA total scores were substantial in both the illiterate subjects (r=0.837, p<.001) and the literate subjects(r=0.802, p<.001). The correlation between the S-LICA and LICA was very high (r=0.989, p<.001). The area under the curve of the receiver operating characteristic was 0.999 for the literate subjects and 0.985 for the illiterate subjects. The sensitivity and specificity of the S-LICA for a diagnosis of dementia were 97% and 96% at the cutoff point of 72 for the literate subjects, and 96% and 93% at the cutoff point of 68 for the illiterate subjects, respectively.

Conclusions

Our results indicate that the S-LICA is a reliable and valid instrument for quick evaluation of patients with dementia in both illiterate and literate elderly populations.

Introduction

In 2008, there were 796 million illiterate adults, which represents approximately 17% of the world's adult population.1 The illiteracy rate remains high among the elderly in South Korea, especially in rural areas. In Yonchon, a Korean rural agricultural area, 26.4% of the elderly are reported to be illiterate, and illiteracy has been associated with a higher risk of Alzheimer's disease.2 Neuropsychological testing is important for the diagnosis and follow-up of dementia, and it may help during consultations on patient care. However, it is difficult to evaluate the cognitive function of illiterate persons using neuropsychological tests that require reading and writing skills or that require individuals to copy a complex figure with a pencil.
The recently developed Literacy-Independent Cognitive Assessment (LICA) is a neuropsychological test battery that is useful for the diagnosis of dementia in both illiterate and literate elderly populations.3 The LICA has 13 subtests that do not require reading, writing, or drawing with a pencil, and that are applicable to most illiterate elderly persons. The sensitivity and specificity of the LICA for the diagnosis of dementia are 91.9% and 91.8%, respectively, for literate subjects and 96.2% and 91.1%, respectively, for illiterate subjects. The LICA is divided into two parts: verbal memory tests and tests for other cognitive domains. The LICA performed well in discriminating Clinical Dementia Rating (CDR) stages and showed good concurrent validity with the Korean version of the Mini-Mental State Examination (K-MMSE) in both literate and illiterate elderly populations.4,5
However, it takes many dementia patients more than 40 minutes to complete the LICA, and some of these people only have an attention span of up to 30 minutes; a skilled clinician may thus be needed to ensure that the subject remains motivated to complete the test. Also, lengthy tests are more costly and can be stressful to the subjects.6 Individuals undergoing dementia assessment are usually older and relatively frail, and they may exhibit some cognitive deterioration. For these individuals, lengthy testing is both physically and mentally tiring. In particular, illiterate elderly individuals fear that they will perform poorly in the tests, which may lead them to become nervous or refuse to complete them. Therefore, there is a need for a shorter form of the LICA with a smaller number of subtests in order to reduce testing time.
The aims of the present study were to use data from the LICA validation study to select a subset of items for an optimal short form of the LICA (S-LICA) and to validate this short form in a new sample using a diagnosis of dementia as the validity standard.

Methods

Development of the S-LICA

The LICA consists of a story recall test, a word list recall test, stick construction, a visuospatial span test, the Digit Stroop test, a money calculation test, a visual recognition test, an animal fluency test, confrontation naming, and the Color and Object Recognition Test (CORT). The LICA was divided into two clearly separated factors in the validation study.3 The first factor included "tests for other cognitive domains except verbal memory" and comprised stick construction, the visuospatial span test, the Digit Stroop test, the money calculation test, the visual recognition test, the animal fluency test, confrontation naming, and the CORT. The second factor included "verbal memory tests" and comprised the story recall test and the word list recall test. We excluded the story recall test of the LICA in the S-LICA because it showed lower test-retest reliability compared to the word list recall test in the validation study.3 When some shy individuals were unable to remember verbs but remembered nouns in the sentences of the story recall test, they refused to say the words that they remembered. A previous study found that the word list learning test was a more useful tool for examining verbal memory function for older adults in Korea than the story recall test.7 The visuospatial span test and the money calculation test were also excluded because very low scores were obtained by some illiterate controls. Therefore, the finally developed S-LICA comprised eight subtests with total scores ranging from 0 to 100 (Table 1). We were able to evaluate verbal and visual memory, language, visuoconstruction, executive function, and semantics of objects with the S-LICA.

Subjects

The S-LICA, K-MMSE, and CDR were administered to 108 patients with dementia (41 males and 67 females) and 188 normal controls (46 males and 142 females). One illiterate patient with dementia could not perform the Digit Stroop test because she was not able to read the numerals '1', '2', and '3'. Two illiterate patients with dementia received scores of less than 10 points on the K-MMSE and a CDR score of 3 points. Three of the normal subjects and four of the dementia patients were aged <60 years. After excluding 10 subjects, 101 patients with dementia and 185 normal controls were included in this study (Fig. 1). The severity of dementia was assessed using the CDR Scale. Twelve patients (1 illiterate and 11 literate) were rated as CDR stage 0.5, 68 (25 illiterate and 43 literate) as stage 1, and 21 as stage 2.
The patients with dementia fulfilled the criteria of the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition.8 They were recruited consecutively among outpatients from the memory clinics of four university hospitals. The normal controls consisted of community-dwelling elderly individuals and the spouses of patients. The controls were aged ≥60 years, did not have subjective memory complaints or any of 28 diseases, and did not have a history suggestive of a decrease in cognitive function (stroke or transient ischemic attack, seizures, Parkinson's disease, multiple sclerosis, cerebral palsy, Huntington's disease, encephalitis, meningitis, brain surgery, vascular surgery of the brain, diabetes requiring insulin control, improperly managed hypertension, cancer diagnosed within the previous 3 years excluding skin cancer, shortness of breath while sitting still, use of oxygen at home, heart attack with changes in memory, walking, or solving problems lasting at least 24 hours afterwards, kidney dialysis, liver disease, hospitalization for mental or emotional problems in the previous 5 years, current use of medications for mental or emotional problems, alcohol consumption greater than 3 drinks daily, drug abuse in the previous 5 years, treatment for alcohol abuse in the previous 5 years, unconsciousness for more than 1 hour other than during surgery, overnight hospitalization due to head injury, illness causing a permanent decrease in memory or other mental functions, trouble with vision that prevents reading ordinary print even with glasses, or difficulty understanding conversations because of hearing even with a hearing aid).9 The controls also had scores that were at least one standard deviation above the mean scores of the respective age- and education-matched population on the K-MMSE and an average score of 0.42 or less on the Korean Instrumental Activities of Daily Living (K-IADL).5,10 It has been demonstrated that this score discriminates dementia from cognitively healthy elderly subjects. The K-IADL score ranges from 0 to 3, with lower scores indicating better function.
Subjects were asked to read aloud two sentences: "Young-Hee was thirsty due to physical exercise. She opened a refrigerator". They were then asked to write what would happen next. Subjects who could read the sentences and write an appropriate response were judged to be literate. Demented patients were stratified into a literate or illiterate group according to caregiver reports of their premorbid reading and writing ability. Subjects who were illiterate due to learning and developmental difficulties were excluded.

Validation study of the S-LICA

The K-MMSE was administered to all of the participants to evaluate its concurrent validity with the S-LICA. The S-LICA and K-MMSE were administered on the same day. The test administrators were blinded to the dementia diagnosis of the participants when administering the two questionnaires. Using a random sampling method, 116 controls were matched with 97 demented patients by age, gender, and education. This method permitted evaluation of the sensitivity and specificity of diagnosing dementia using receiver operating characteristic (ROC) curve analysis. The association between the S-LICA and the original LICA was evaluated by administering the S-LICA to 41 normal elderly subjects 1 month after administration of the LICA. To evaluate test-retest reliability, the S-LICA was administered twice with an interval of 28±4 days (mean±SD) to 37 subjects (6 illiterate and 15 literate normal elderly subjects, and 3 illiterate and 13 literate subjects with dementia). The study protocol and informed consent form were approved by the institutional review board. Literate subjects gave written informed consent to participate in the study, and illiterate subjects gave verbal consent.

Statistical analysis

Student's t-test was used to test for differences in age and education between the control group and the dementia group. The difference in the gender ratio between the two groups was evaluated using the chi-square test. Cronbach's coefficient alphas and item-total correlation coefficients were generated to examine the internal consistency of the S-LICA. Pearson correlation coefficients were generated to evaluate test-retest reliability. To evaluate concurrent validity, S-LICA total scores were compared with the K-MMSE scores and the LICA using Pearson's correlation coefficients. An ROC curve was used to examine the sensitivity and specificity of the S-LICA. Statistical analyses were performed with SPSS 18.0 for Windows (SPSS, Chicago, IL, USA), and the level of statistical significance was set at p<0.05 for all analyses.

Results

Demographic characteristics

This study was conducted between March 2009 and October 2009. The demographic characteristics of the subjects are listed in Table 2. The years of education did not differ significantly between the subjects with dementia and the normal subjects. However, the proportion of men was lower among the normal subjects (25%) than among those with dementia (39%, p<.001).

Reliability

The Cronbach's coefficient alpha was 0.92 for illiterate subjects and 0.94 for literate subjects. The item-total correlation ranged from 0.63 to 0.81 and was statistically significant (p<0.01) (Table 3). The test-retest reliability of the S-LICA total score was high (r=0.94, p<.001), and the subtests had high test-retest reliabilities (r=0.68-0.87, p<.01) (Table 3).

Validity

The correlation between the total scores of the shortened version and the original version of the LICA was very strong (r=0.99, p<.001). The linear correlation between the K-MMSE and S-LICA total scores was substantial for both the illiterate (r=0.84, p<.001) and literate (r=0.80, p<.001) subjects.
The normal controls were matched to the dementia patients to evaluate the sensitivity and specificity of the diagnosis of dementia; there were no significant differences in age, education, or gender ratio between patients with dementia and controls for either literacy group (Table 2).
The ROC curve was used to determine the degree to which the S-LICA screened dementia. The area under the curve (AUC) of the ROC curve was 0.999 (95% confidence interval=0.980-0.999, SE=0.005) for the literate subjects and 0.985 (95% confidence interval=0.975-0.994, SE=0.007) for the illiterate subjects. The sensitivity and specificity of the S-LICA for a diagnosis of dementia were 97% and 96%, respectively, with a cutoff of 72 for the literate subjects, and 96% and 93%, respectively, with a cutoff of 68 for the illiterate subjects (Fig. 2).

Application time

The total application time of the S-LICA was 18.1±9.5 minutes for the literate normal controls (n=24), 18.6±8.7 minutes for the illiterate normal controls (n=22), 21.2±15.4 minutes for the literate dementia patients (n=33), and 22±17.5 minutes for individuals who were illiterate with dementia (n=19).

Discussion

The S-LICA was found to be a valid and reliable instrument for the detection of dementia in both illiterate and literate elderly persons. The Cronbach's coefficient alpha for the S-LICA was 0.92 for illiterate subjects and 0.94 for literate subjects. The internal consistency of the S-LICA was comparable to that of the LICA (Cronbach's coefficient alpha of the LICA: 0.908 for illiterate subjects and 0.912 for literate subjects).3 The test-retest reliabilities after a 1-month interval for the total S-LICA score (r=0.94) and the subtests (r=0.68-0.87) were higher than those for the LICA total score (r=0.92) and the LICA subtests (r=0.50-0.86). The lowest test-retest reliability (r=0.50) was observed for the delayed story recall test among the LICA subtests.3 Since the story recall test was excluded from the S-LICA, the test-retest reliability of the S-LICA may be improved compared with the LICA.
The concurrent validity of the S-LICA with the LICA was superior (r=0.99), indicating that the S-LICA could be used instead of the LICA. The linear correlation between the K-MMSE and S-LICA total scores was stronger for both the illiterate (r=0.84) and literate (r=0.80) subjects relative to those between the LICA and K-MMSE (r=0.75 for the illiterate subjects and r=0.76 for the literate subjects).3 Relatively low correlation coefficients (below 0.55) with the K-MMSE were observed for immediate and delayed story recall, forward visuospatial span, and CORT among the LICA subtests. The story recall and visuospatial span tests were excluded from the S-LICA, and so the concurrent validity of the S-LICA with the K-MMSE might be improved in comparison with the LICA.
The diagnostic accuracy of the S-LICA was high (AUC=0.985) for distinguishing illiterate patients with dementia from those with cognitively normal elderly subjects, and was comparable that of the LICA (AUC=0.985). The diagnostic accuracy for literate subjects was higher for the S-LICA (AUC=0.999) than for the LICA (AUC=0.985).3 The sensitivity and specificity of the S-LICA for a diagnosis of dementia were 96% and 93%, respectively, with a cutoff score of 70 for the illiterate subjects, and were comparable to those for the LICA (sensitivity of 96.2% and specificity of 91.1% for a cutoff of 154.5).3 The sensitivity (97%) and specificity (96%) of the S-LICA for a diagnosis of dementia with a cutoff score of 72 for the literate subjects were superior to those of the LICA (sensitivity of 91.9% and specificity of 91.8% for a cutoff of 186.0).3 These results allow us to conclude that the S-LICA has diagnostic accuracy that is comparable to the LICA for a diagnosis of dementia among both literate and illiterate elderly populations.
In comparison with the original LICA, a strength of the shortened version is that it takes only approximately 20 minutes to complete for both literate and illiterate subjects. This may reduce the cost of the neuropsychological test and increase the likelihood of obtaining reliable data from elderly subjects by enhancing the cooperation of those who may have difficulty sustaining attention for longer periods of time. The S-LICA also has the potential to be used as a screening test for dementia because of its short application time and high sensitivity. The scores on the S-LICA range from 0 to 100, with higher scores indicating better performance. Therefore, the S-LICA scores may be interpreted more easily than LICA, for which the possible score ranges from 0 to 300. The S-LICA can evaluate cognitive domains, such as verbal and visual memory, visuoconstruction, executive function, language, and semantics, which are associated with early cognitive impairment in dementia and are essential for classifying the subtypes of mild cognitive impairment. The S-LICA is also useful for following up cognitive function in patients because it presents a total score.
There is little difference in the cutoff levels of the S-LICA for the diagnosis of dementia between the literate and illiterate subjects compared with the LICA. This suggests that literacy affects the S-LICA less than the LICA. Commonly used screening tests, including the Clock-Drawing Test, the General Practitioner Assessment of Cognition, the Mini-Cog, the Montreal Cognitive Assessment, the Seven-Minute Test, the Rowland Universal Dementia Assessment Scale, and the MMSE, require reading, writing, or drawing, which may affect the performance among illiterates and individuals with low educational levels.12-20 Therefore, a brief test such as the S-LICA may be considered to evaluate cognitive function in illiterate individuals and elderly individuals with low educational levels.

Study limitations

This study was subject several limitations. First, the small numbers of subjects could have introduced selection or other bias. Second, only a clinic-based patient population was included, as in the LICA study. Third, the analysis was not conducted according to subtypes of dementia. Future studies should compare aspects of cognitive impairment among various subtypes of dementia using the S-LICA. Finally, patients with a CDR rating of >stage 3 were not included in this study, and hence future studies should also be conducted with patients in these stages of dementia. In addition, the illiterate female group was larger than the illiterate male group. However, data from the United Nations Educational, Scientific, and Cultural Organization show that worldwide, the prevalence of illiteracy is also greater for women than for men.1

Conclusion

The findings of this study show that the S-LICA is a valid and reliable instrument for detecting dementia in both illiterate and literate elderly individuals. Its diagnostic accuracy appears to be comparable to that of the LICA, while its test-retest reliability and concurrent validity are superior. The S-LICA also has the strengths of a shorter application time and easier interpretation of the total score compared to the LICA.

Figures and Tables

Fig. 1
Study profile.
jcn-9-111-g001
Fig. 2
The receiver operating characteristic curves of the S-LICA used to make a diagnosis of dementia in the illiterate (A) and literate (B) subjects. S-LICA: short form of the Literacy-Independent Cognitive Assessment.
jcn-9-111-g002
Table 1
Outline of the short form of the Literacy-Independent Cognitive Assessment (LICA)3
jcn-9-111-i001

CORT: Color and Object Recognition Test.

Table 2
Demographic characteristics of the subjects
jcn-9-111-i002

*p<0.001 versus controls by chi-square test.

ROC: receiver operating characteristic.

Table 3
Corrected item-total correlations and test-retest reliabilities of the short form of the LICA (S-LICA)
jcn-9-111-i003

*p<.01, **p<.001.

LICA: Literacy-Independent Cognitive Assessment.

Acknowledgements

This study was supported by grants of the Korea Healthcare technology R&D Project, the Ministry of Health and Welfare, Republic of Korea (A102065).

Notes

The authors have no financial conflicts of interest.

References

1. United Nations Educational, Scientific and Cultural Organization. EFA Global Monitoring Report 2011. 2011. cited 2012 Sep 11. Paris: United Nations Educational, Scientific and Cultural Organization;65. Available from: URL: http://unesdoc.unesco.org/images/0019/001907/190743e.pdf.
2. Lee JY, Chang SM, Jang HS, Chang JS, Suh GH, Jung HY, et al. Illiteracy and the incidence of Alzheimer's disease in the Yonchon county survey, Korea. Int Psychogeriatr. 2008. 20:976–985.
crossref
3. Choi SH, Shim YS, Ryu SH, Ryu HJ, Lee DW, Lee JY, et al. Validation of the literacy independent cognitive assessment. Int Psychogeriatr. 2011. 23:593–601.
crossref
4. Morris JC. The Clinical Dementia Rating (CDR): current version and scoring rules. Neurology. 1993. 43:2412–2414.
5. Kang Y. A normative study of the Korean-mini mental state examination (K-MMSE) in the elderly. Korean J Psychol Gen. 2006. 25:1–12.
6. Teng EL, Manly JJ. Neuropsychological testing: helpful or harmful? Alzheimer Dis Assoc Disord. 2005. 19:267–271.
7. Baek MJ, Kim HJ, Kim S. Comparison between the story recall test and the word-list learning test in Korean patients with mild cognitive impairment and early stage of Alzheimer's disease. J Clin Exp Neuropsychol. 2012. 34:396–404.
crossref
8. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders: DSM-IV. 1994. 4th ed. Washington, DC: American Psychiatric Association.
9. Christensen KJ, Multhaup KS, Nordstrom S, Voss K. A cognitive battery for dementia: development and measurement characteristics. Psychol Assess. 1991. 3:168–174.
crossref
10. Kang SJ, Choi SH, Lee BH, Kwon JC, Na DL, Han SH. Korean Dementia Research Group. The reliability and validity of the Korean Instrumental Activities of Daily Living (K-IADL). J Korean Neurol Assoc. 2002. 20:8–14.
11. Shulman KI. Clock-drawing: is it the ideal cognitive screening test? Int J Geriatr Psychiatry. 2000. 15:548–561.
crossref
12. Brodaty H, Pond D, Kemp NM, Luscombe G, Harding L, Berman K, et al. The GPCOG: a new screening test for dementia designed for general practice. J Am Geriatr Soc. 2002. 50:530–534.
crossref
13. Borson S, Scanlan J, Brush M, Vitaliano P, Dokmak A. The mini-cog: a cognitive 'vital signs' measure for dementia screening in multi-lingual elderly. Int J Geriatr Psychiatry. 2000. 15:1021–1027.
crossref
14. Nasreddine ZS, Phillips NA, Bédirian V, Charbonneau S, Whitehead V, Collin I, et al. The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc. 2005. 53:695–699.
crossref
15. Lee JY, Lee DW, Cho SJ, Na DL, Jeon HJ, Kim SK, et al. Brief screening for mild cognitive impairment in elderly outpatient clinic: validation of the Korean version of the Montreal Cognitive Assessment. J Geriatr Psychiatry Neurol. 2008. 21:104–110.
crossref
16. Solomon PR, Hirschoff A, Kelly B, Relin M, Brush M, DeVeaux RD, et al. A 7 minute neurocognitive screening battery highly sensitive to Alzheimer's disease. Arch Neurol. 1998. 55:349–355.
crossref
17. Storey JE, Rowland JT, Basic D, Conforti DA, Dickson HG. The Rowland Universal Dementia Assessment Scale (RUDAS): a multicultural cognitive assessment scale. Int Psychogeriatr. 2004. 16:13–31.
crossref
18. Folstein MF, Folstein SE, McHugh PR. "Mini-mental state". A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975. 12:189–198.
19. Ismail Z, Rajji TK, Shulman KI. Brief cognitive screening instruments: an update. Int J Geriatr Psychiatry. 2010. 25:111–120.
crossref
20. Ridha B, Rossor M. The mini mental state examination. Pract Neurol. 2005. 5:298–303.
crossref
TOOLS
Similar articles