PDF
Citation
Print
INTRODUCTION
With the rapid increase in the elderly population in Korea, dementia has emerged as a major health problem.1 The Korean government has instituted a national screening program for transitional ages (NSPTA) that includes screening for cognitive dysfunction at ages 66, 70, and 74 years, and which has been covered by the National Health Insurance since 2007.2 The primary screening tool in the NSPTA to detect cognitive dysfunction has been The Korean Dementia Screening Questionnaire-Prescreening (KDSQ-P), while The Korean Dementia Screening Questionnaire-Cognition (KDSQ-C) has been used for secondary screening.3 Although the KDSQ-P and KDSQ-C were designed to be completed by a reliable informant, these questionnaires are frequently completed by the patient him- or herself. Since 2018, the NSPTA has been integrated into the periodic general health screening program and the KDSQ-P is no longer used; the KDSQ-C is used as a one-step screening tool. The KDSQ-C is a semi-structured questionnaire that includes 15 questions that assess memory impairment, other cognitive impairments including language impairments, and the ability to perform complex tasks in daily life. The KDSQ-C has answer options of ‘no,’ ‘sometimes,’ and ‘often,’ which are scored as 0, 1, 2, respectively. A score of 6 is considered a valid threshold to identify dementia.4
Several other screening tools for dementia have been also used in Korea. The Alzheimer Disease 8 (AD8), developed by Galvin et al.5 in 2005, is useful to distinguish between normal cognitive function (Clinical Dementia Rating [CDR]6 of 0), mild cognitive impairment (MCI), and very early dementia (CDR of 0.5). It consists of eight questions about changes in memory (answered with ‘yes’ or ‘no’), orientation, problem-solving ability, and activities of daily living.5 Although the AD8 was originally developed to be completed by an informant other than the patient, it may also be completed by patients themselves.78 A Korean version of the AD8 has also been developed, and a score of 2 was considered a valid threshold to distinguish dementia.9 Using a cut-off score of 2, the AD8 showed a sensitivity of 0.74 and specificity of 0.86 for discriminating normal cognition from MCI and mild dementia.10
The Subjective Memory Complaints Questionnaire (SMCQ) is a patient-completed screening test for subjective cognitive decline (SCD) developed by Youn et al.11 in 2009. The authors suggest that the SMCQ can discriminate elderly patients with dementia from those without dementia.11 The questionnaire consists of 14 items including subjective reports of general memory and everyday memory. Each question is answered with ‘yes’ or ‘no’ and scored as 1 or 0 points, and a score of 6 is used as the cut-off. The SMCQ has shown a sensitivity of 0.75 and specificity of 0.69.11 The validity of the questionnaire when completed by a caretaker of the patient has not been assessed.
Elderly patients who undergo a national health screening often do this in the absence of a caregiver, which means that the KDSQ-C questionnaire for cognitive dysfunction is often performed by the patient although they have originally been designed to be completed by an informant.4 Therefore, concerns about KDSQ-C questionnaires completed by patients have been raised. The SMCQ, which was originally developed as a self-report questionnaire, has been suggested as an alternative to the KDSQ-C, as has the AD8.
Therefore, the current study aims to examine these concerns by comparing the diagnostic ability of screening questionnaires for cognitive impairment when completed by the patient or an informant during periodic national health screening visits. Specifically, we aimed to evaluate whether the SMCQ or the AD8 is superior to the KDSQ-C and can be used as its replacement.
METHODS
Participants
From August 2017 to April 2018, a total of 420 participants (200 healthy controls, 50 with MCI, 120 with Alzheimer disease [AD], and 50 with other dementias) aged 65 years and above and their informants were recruited from 11 hospitals across Korea, located in Seoul (7), Gyeonggi (3), and Busan (1). These participants were recruited from the department of neurology or the psychiatry outpatient clinic at each hospital or from regional dementia centers in local districts (Mapo-gu, Yangcheon-gu, Gangseo-gu) of Seoul. All participants were examined by an experienced neurologist or psychiatrist and were divided into three subgroups: normal cognition, MCI, and dementia. Participants with normal cognition were defined as cognitively and functionally normal and independent, and fulfilled the health screening exclusion criteria by Christensen et al.12 The normal group had a Mini-Mental State Examination (MMSE) score ≥ 1.0 standard deviation (SD) below the norm. MCI was diagnosed based on the criteria of Peterson.1314 The specific inclusion criteria for MCI were as follows: 1) patient, informant or both reported cognitive decline, 2) cognitive impairment (< 1.5 SDs below age and education-adjusted norms) in ≥ 1 domain (executive function, memory, language, or visuospatial) on standard neuropsychological tests, 3) normal functional activities, and 4) no presence of dementia according to criteria of the Diagnostic and Statistical Manual of Mental Disorders, Fourth edition, Text Revision (DSM-IV-TR).15
Dementia was categorized as either AD, vascular dementia, frontotemporal dementia (FTD), or dementia with Lewy bodies (DLB). AD was diagnosed based on the probable AD criteria proposed by the National Institute of Neurological and Communicative Disorders and Stroke and AD and Related Disorders Association16 as well as DSM-IV-TR.15 Vascular dementia was diagnosed based on the DSM-IV-TR criteria15 for vascular dementia. FTD was diagnosed based on ‘research criteria for frontotemporal dementia’.17 DLB was diagnosed with the probable DLB criteria proposed in the third report of the DLB consortium.18 Recruitment was limited to patients with very mild to moderate dementia with a CDR of 0.5, 1, or 2.
Individuals with structural or laboratory testing abnormalities that could lead to cognitive decline were excluded. These abnormalities included head injury that resulted in loss of consciousness or cognitive impairment for more than an hour; a history of cerebral hemorrhage or subarachnoid hemorrhage; a space-occupying brain lesion; cognitive impairment associated with neurosyphilis; HIV infection, thyroid abnormalities, vitamin B12 or folate deficiency; and history of metabolic encephalopathy. All individuals with Axis I psychiatric disorders, including depression, anxiety, and obsessive-compulsive disorder, were excluded. Individuals with a physical illness or disorder that could interfere with the clinical study, including hearing or vision loss, aphasia, severe cardiac failure, severe respiratory illnesses, uncontrolled diabetes, malignancy, or hepatic failure or renal disorders with dialysis, were excluded.
Informants were included if they interacted with the participant three or more days per week and spend more than four hours at each interaction. This ensured that informants understood the participant's condition and were able to complete assessments, including questionnaires.
Clinical evaluations
We examined baseline demographic data including age, gender, years of education, medical history, and family history. All participants underwent a Mini-Mental State Examination-Dementia Screening (MMSE-DS)19 test and completed the short form of the Geriatric Depression Scale20 to evaluate global cognition and depression, respectively. The KDSQ-C, AD8, and SMCQ were completed separately by participants and their informants to evaluate participants' cognitive function and activities of daily living. Twenty patients with AD, 20 patients with MCI, 40 normal participants, and their informants were asked to complete the same questionnaires one month later to determine their test-retest reliability. The order of administering the three questionnaires was randomly selected. Follow-up questionnaires were administered in the same order as the initial evaluation.
Statistical analysis
Categorical variables are presented as percentages, and continuous variables are presented as mean ± SD or median (interquartile range). To compare groups, Student's t-tests were used for continuous variables with a normal distribution, the Wilcoxon rank-sum test was used for continuous variables without a normal distribution, and the χ2 test or Fisher's exact test were used for categorical variables. The sensitivity and specificity of each questionnaire and the MMSE-DS for diagnosing dementia were calculated with a receiver operating characteristic (ROC) curve. Areas under the curve (AUCs) with 95% confidence intervals were generated to assess the diagnostic ability of each screening questionnaire. The AUCs for each instrument were compared using the DeLong method. The AUCs of the questionnaires were considered to be statistically significantly different when the P value was less than 0.008 after applying the Bonferroni correction for multiple comparisons. The optimal cut-off score for each questionnaire was selected when Youden's index was maximized by the ROC curve. Test-retest reliability was assessed with the intraclass correlation coefficient (ICC) and a two-way mixed effects model. Pearson's correlation coefficient (r) was also calculated. Correlations between participant and informant questionnaire scores were determined with Spearman's correlation coefficients. Fisher's r-to-z transformation analysis was used to assess a statistically significant difference between the two correlation coefficients. Significance was set at P < 0.05. Statistical analyses were performed with SAS 9.4 (SAS Inst., Cary, NC, USA).
RESULTS
Demographic characteristics
Of the 420 participants, 233 (55.5%) were women (Table 1). The mean ± SD for participant age at the time of assessment was 75.3 ± 3.9 years (range, 60–93 years), and the mean years of education was 9.1 ± 4.9 years (range, 0–20 years). The study sample consisted of 200 normal participants, 50 participants with MCI, and 170 participants with dementia. In the dementia group, 120 (70.6%) participants were diagnosed with AD, 22 (12.9%) with vascular dementia, 14 (8.2%) with DLB, and 14 (8.2%) with FTD. Eighty-two participants (19.5%) had a family history of dementia: 12.5% in the normal group, 46.0% in the MCI group, and 20.0% in the dementia group. The MCI and dementia groups were significantly more likely to have a family history of dementia than the normal group (P < 0.001) (Table 1). Other characteristics are presented in Table 1.
Table 1
Baseline demographic characteristics of study subjects
Comparison of questionnaire scores by group
Table 2 shows the MMSE-DS and questionnaire scores for the normal and cognitive impairment groups evaluated by participants and informants. The highest mean MMSE-DS score was found in the normal group (27.3 ± 1.9 points), followed by the MCI group (24.3 ± 3.2 points), and the dementia group (18.5 ± 5.4 points). The mean ± SD scores for participant-completed KDSQ-C (p-KDSQ-C) were 3.9 ± 3.5 in the normal group, 6.4 ± 5.0 in the MCI group, and 9.3 ± 7.4 in the dementia group. The participant-completed AD8 (p-AD8) scores were 1.0 ± 1.5 in the normal group, 2.5 ± 2.0 points in the MCI group, and 3.7 ± 2.6 points in the dementia group. The participant-completed SMCQ (p-SMCQ) scores were 2.9 ± 2.6 in the normal group, 5.2 ± 3.0 in the MCI group, and 5.9 ± 4.0 in the dementia group. Participant-completed questionnaire scores tended to be lowest in the normal group, followed by the MCI and dementia groups. Informant-completed questionnaire scores also tended to be lowest in the normal group, followed by the MCI and dementia groups. Informant-completed questionnaire scores in the MCI and dementia groups were higher than participant-completed questionnaire scores.
Table 2
Comparison of questionnaire scores by group
P for differences were determined by using the Wilcoxon rank sum test.
MCI = mild cognitive impairment, SD = standard deviation, IQR = interquartile range, MMSE-DS = Mini-Mental State Examination-Dementia Screening, p-KDSQ-C = participant-completed Korean Dementia Screening Questionnaire-Cognition, p-AD8 = participant-completed Alzheimer Disease 8, p-SMCQ = participant-completed Subjective Memory Complaints Questionnaire, i-KDSQ-C = informant-completed Korean Dementia Screening Questionnaire-Cognition, i-AD8 = informant-completed Alzheimer Disease 8, i-SMCQ = informant-completed Subjective Memory Complaints Questionnaire.
The sensitivity, specificity, and AUC of the MMSE-DS, AD8, KDSQ-C, and SMCQ
ROC curves were generated to measure the effectiveness of each instrument in classifying the normal vs. dementia groups (Fig. 1). The AUC for MMSE-DS was 0.95. The p-AD8 had an AUC of 0.80, which was higher than that of the p-KDSQ-C (0.75) and p-SMCQ (0.73). For screening of the normal and dementia groups, the p-KDSQ-C had a sensitivity of 0.62 and specificity of 0.77 using a cut-off score of 6. The p-AD8 had a sensitivity of 0.71 and specificity of 0.75 using a cut-off score of 2, and a sensitivity of 0.61 and specificity of 0.89 using an optimal cut-off score of 3. The p-SMCQ had a sensitivity of 0.49 and specificity of 0.84 using a cut-off score of 6 and a sensitivity of 0.67 and specificity of 0.71 using an optimal cut-off score of 4 (Table 3). The AUCs of the informant-completed AD8 (i-AD8), informant-KDSQ-C (i-KDSQ-C), and informant-SMCQ (i-SMCQ) were 0.93, 0.92, and 0.92, respectively. These AUCs were higher than those of the participants. The i-KDSQ-C had the best combination of sensitivity (0.85) and specificity (0.79) using a cut-off score of 6. The i-AD8 had the best combination of sensitivity (0.90) and specificity (0.81) using a cut-off score of 2. The i-SMCQ had the best combination of sensitivity (0.88) and specificity (0.83) using an optimal cut-off score of 5. Both the known and optimal cut-off scores had high sensitivity and specificity (Table 4).
Fig. 1
ROC curve (normal vs. dementia). (A) ROC curve, participants. (B) ROC curve, informants.
ROC = receiver operating characteristic, MMSE-DS = Mini-Mental State Examination-Dementia Screening, KDSQ-C = Korean Dementia Screening Questionnaire-Cognition, AD8 = Alzheimer Disease 8, SMCQ = Subjective Memory Complaints Questionnaire.
Table 3
Sensitivity, specificity of each instrument completed by participants at different cut-off scores for discriminating dementia (normal vs. dementia)
AUC = area under the curve, CI= confidence interval, MMSE-DS = Mini-Mental State Examination-Dementia Screening, p-KDSQ-C = participant-completed Korean Dementia Screening Questionnaire-Cognition, p-AD8 = participant-completed Alzheimer Disease 8, p-SMCQ = participant-completed Subjective Memory Complaints Questionnaire.
aOptimal cut-off by Youden's index.
Table 4
Sensitivity, specificity of each instrument completed by informants at different cut-off scores for discriminating dementia (normal vs. dementia)
Comparison of AUCs
For discriminating dementia from the normal group with participant-completed questionnaires, the AUC of the MMSE-DS had significantly better results than did all other questionnaires (P < 0.001). The AUC for the p-AD8 was significantly higher than the AUCs for the p-KDSQ-C and p-SMCQ (P < 0.005). The AUCs for the MMSE-DS and all three questionnaires completed by informants did not differ significantly (P > 0.008) (Table 5).
Table 5
Comparison of AUCs (normal vs. dementia)
Test-retest reliability
At the 1-month follow-up, the ICCs of the participant-completed questionnaires were 0.90, 0.86, and 0.88 for the p-KDSQ-C, p-AD8, and p-SMCQ, respectively. For the informant-completed questionnaires, the ICCs were 0.93, 0.93, and 0.95 for the i-KDSQ-C, i-AD8, and i-SMCQ, respectively. The test-retest reliability of informant-completed questionnaires was higher than those completed by participants (Table 6).
Table 6
Test-retest reliability
Correlation of scores between participants and informants
Table 7 shows the correlation between the scores of the MMSE-DS and the other questionnaires using Spearman's correlation. The MMSE-DS was negatively correlated with each questionnaire, and the questionnaires were positively correlated with each other. For participants, the p-KDSQ-C and p-SMCQ had the highest correlation. The p-SMCQ and p-AD8, and p-KDSQ-C and p-AD8 also had high correlations. For informants, the i-KDSQ-C and i-SMCQ had the highest correlation. The i-SMCQ and i-AD8, and i-KDSQ-C and i-AD8 also had a high correlation. Informant- and participant-completed scores were moderately correlated. The MMSE-DS correlated with the p-KDSQ-C, p-AD8, and p-SMCQ to a lesser degree than with the i-KDSQ-C, i-AD8, and i-SMCQ. There were statistically significant differences between the correlation coefficient of the MMSE-DS and the correlation coefficients of each of the three other questionnaires as evaluated by Fisher's r-to-z transformation analyses (Table 8). When the correlations between the MMSE-DS and the three questionnaires were analyzed by subgroup, the correlation coefficients of the normal group and the MCI group were lower compared to the total sample, although not statistically significant. However, the dementia subgroup showed similar results to those of the total sample (data not shown).
Table 7
Correlation of MMSE-DS, KDSQ-C, AD8, and SMCQ between participants and informants
Spearman correlation matrix.
MMSE-DS = Mini-Mental State Examination-Dementia Screening, KDSQ-C = Korean Dementia Screening Questionnaire-Cognition, p-KDSQ-C = participant-completed Korean Dementia Screening Questionnaire-Cognition, AD8 = Alzheimer Disease 8, p-AD8 = participant-completed Alzheimer Disease 8, SMCQ = Subjective Memory Complaints Questionnaire, p-SMCQ = participant-completed Subjective Memory Complaints Questionnaire, i-KDSQ-C = informant-completed Korean Dementia Screening Questionnaire-Cognition, i-AD8 = informant-completed Alzheimer Disease 8, i-SMCQ = informant-completed Subjective Memory Complaints Questionnaire.
Table 8
Results of Fisher's r-to-z transformation for significance of difference between correlation coefficients (n = 420)
The Fisher's r-to-z transformation (z score) was used to determine significant differences between correlations.
CI = confidence interval, KDSQ-C = Korean Dementia Screening Questionnaire-Cognition, MMSE-DS = Mini-Mental State Examination-Dementia Screening, AD8 = Alzheimer Disease 8, SMCQ = Subjective Memory Complaints Questionnaire.
DISCUSSION
To our knowledge, this is the first study to compare the diagnostic ability of three questionnaires for dementia evaluation completed by participants and their informants. Regarding informant-completed questionnaires, the AD8, KDSQ-C, and SMCQ showed highly accurate discriminability for dementia, with no difference in discriminability between the three questionnaires. Among participant-completed questionnaires, all three questionnaires showed moderately accurate discriminability for dementia, although they were lower than those of the informant-completed questionnaires. The AD8 had a higher discriminability for dementia than did the KDSQ-C and SMCQ, whose discriminability did not differ from each other. The MMSE-DS correlated negatively with informant- and participant-completed questionnaires, although informant-completed questionnaires showed a higher correlation with the MMSE-DS than participant-completed questionnaires.
The AD8 and KDSQ-C assess not only memory and but also other cognitive functions. The AD8 is based on the CDR, but customized for AD.5 The SMCQ consists of memory-related questions only, and, thus, it may not be able to identify non-AD with non-amnestic symptoms.11 The KDSQ-C uses responses of ‘no’, ‘sometimes’, and ‘often’,4 while the AD8 and SMCQ use ‘yes’ and ‘no’. Three options can provide more accuracy than two options, but can make questions more complex and difficult to answer. Dementia patients, especially, may have difficulty selecting the best answer. Therefore, the AD8 may have better results than the other questionnaires when completed by participants. The present findings are compatible with previous research.7
When comparing questionnaire results between participants and informants, informant-completed questionnaires were more reliable than participant-completed questionnaires. This finding is in line with previous studies, showing that informant-based measurement is more useful in early screening for cognition changes.2122 However, questionnaires completed by patients with MCI or mild dementia could be used to screen for cognitive decline. The AD8 was superior to the other questionnaires. Considering that most patients attend health check-ups without an informant, the results of this study have important practical implications.
We presented two cut-off scores: the cut-off score from previous studies4511 and the cut-off score we obtained with Youden's index. Using a different cut-off changes questionnaire sensitivity and specificity. We suggest that using cut-off scores with a higher sensitivity is better, because these questionnaires will be used in screening for dementia. The present study confirmed the known cut-off scores as useful in the KDSQ-C and AD8; however, we recommend a new cut-off score for SMCQ.
When the questionnaires were completed by the informant, all three questionnaires had higher scores than when completed by the participants in the MCI and dementia groups. These findings suggest that the MCI and dementia groups could perceive their cognitive decline and evaluate the deficit, but they had less ability to properly evaluate their symptoms and functions than did informants. In the normal group, participants reported higher scores than did informants, probably because some participants with SCD were included in the normal group.
Medical check-ups often depend on a country's social context and cultural background rather than scientific evidence. Korea's population is aging more rapidly than any other country in the world, became an aged society in 2017, and will be a super-aged society by 2026.23 Dementia has rapidly emerged as a major health problem in Korea.1 This context has influenced the Korean national screening program for cognitive impairment to be implemented for patients over 65 years old.
The present study has public health implications. In cases where informant-report is possible, we recommend the KDSQ-C continue to be used in national medical check-ups. The KDSQ-C is not different from AD8 or SMCQ in discriminability for dementia, and consistent data collection by the same questionnaire is important. In cases where informant-report is not possible, either the patient-completed KDSQ-C or AD8 may be used, although their discriminability is lower than those of informant-completed questionnaires. Thus, it is important to obtain information from the informant, even if only by telephone, when the KDSQ-C is used in national medical check-ups.
The present study has several strengths. It is a multicenter study involving sites across the country. Questionnaires were administered to all participants and informants. The diagnosis was based on clinical symptoms and laboratory findings and not limited to questionnaire results. Test-retest reliability of questionnaires was assessed for both participants and informants. There are also several limitations of the present study. First, participants were recruited from tertiary university hospitals and dementia centers, and may not represent the general population. Second, the normal group may have included participants with SCD, which might have influenced the results of the present study. Third, as AD accounts for the majority (70.6%) of dementia participants in the present study, we were not able to analyze AD and non-AD dementia, respectively, due to the small sample of non-AD dementia patients. Dominance of AD among dementia might have produced better discriminability of AD8 over KDSQ. Fourth, the small number of MCI cases prevented determining the discriminability of the scale for screening of MCI with validity. Finally, we did not perform imaging studies or comprehensive neuropsychological tests for the normal group.
In conclusion, the KDSQ-C should continue to be used in national medical check-ups in cases with an available informant. When an informant is not available, either the KDSQ-C or AD8 may be used. It is important to obtain information from the informant, even if only by telephone, when the KDSQ-C is used in national medical check-ups.
XML Download