This study recruited 176 Chinese-speaking participants who were 60 years or older with reasonable vision, hearing, and ability to communicate verbally from the Department of Neurology, Sichuan Provincial People's Hospital, Chengdu, China. Patients with a history of major depression, schizophrenia, epilepsy, significant head injury, substance abuse, alcoholism, or other severe physical disorders were excluded. Informed consents were obtained from all participants, and the study protocol was approved by the Institutional Ethics Committee (IRB No. 20170032).

The demographic information and clinical profiles including history/informant reports, presentation at interview, general and neurological examinations, neuropsychological examinations, neuroimaging (structural CT/MRI and angiography), and standard dementia screening blood tests (folic acid, vitamin B₁₂, thyroid function, and treponema pallidum antibody) were recorded by two qualified neurologists. The Chinese version of the Clinical Dementia Rating Scale (CDR) was adopted to evaluate the daily and social functioning of the participants.21 The common objects memory test (COMT) is a list-learning task that has been validated across cultures (including Chinese), and it was used to provide objective evidence of memory decline in the diagnoses of MCI.22 The scales, administration instructions, and normative data of COMT are available online. The participants were then divided into three groups: 1) mild dementia group, comprising patients who met the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-V) dementia criteria and had a CDR score of 1; 2) MCI group, comprising patients with memory complaints that were corroborated by an informant; abnormal memory function, as documented by scores for the COMT items (including 5-min recall/recognition and 30-min recall/recognition) falling at least 1.5 SDs below the age- and education-specific norms; a CDR score of 0.5; normal ability to perform the activities of daily living; and not demented, all as proposed by Petersen et al.3 in 1999; and 3) control group, comprising patients with no memory complaints, a CDR score of 0, and a normal ability to perform the activities of daily living.13 Each of the two neurologists was responsible for 50% of the participants and double checked the grouping decisions made by the other. Disputes were resolved by consensus.

Another two researchers, who were blind to the participants' grouping, administered the Chinese versions of MMSE, MoCA (Beijing version), and ACE-III to all participants within 1 week after clinical assessments.81213 The MMSE, MoCA, and ACE-III scores were not used in clinical diagnoses nor influenced by the grouping. The three cognitive tests were administered and scored in accordance with their respective administration instructions. All diagnoses of etiology were made according to the following criteria: National Institute of Neurologic, Communicative Disorders and Stroke-Alzheimer's Disease and Related Disorders Association (NINCDS-ADRDA) criteria for the diagnosis of AD; International Classification of Diseases, Tenth Edition (ICD-10) criteria for the diagnosis of vascular dementia (VD); and DSM-V criteria for diagnoses of frontotemporal dementia (FTD), Lewy-body dementia (LBD), and Parkinson's disease dementia (PDD).

The study of Wang et al.18 and the present study were performed in widely separated geographic locations and the participants differed considerably in education level and linguistic habits. Thus, the Chinese version of ACE-III was redeveloped for use in the present study, as described below.

Firstly, the Chinese version of ACE-III was translated using forward and backward translation methodologies. For forward translation, one bilingual neurologist translated the original instrument into Chinese, and then another bilingual neurologist translated the Chinese version back into the English version. Discrepancies between the two versions were identified and corrected. Three such procedures were performed until the back-translated version was equivalent to the original English version.

Secondly, in order to make the scale easier to understand and follow for the Chinese-speaking participants, changes were made to the following items in the English version: name/address recall and recognition, semantic memory, verbal fluency, and repetition, reading, and comprehension items in the language subtest. Based on the original criteria, a common Chinese name and local address (district, place, street, and number) were used in the memory subtest for recall and recognition. In the recognition part, another two sets of name and local addresses were composed as recognition stimuli. In the semantic memory subtest, the four original questions were replaced with culturally equivalent counterparts as follows: “the current president of the People's Republic of China,” “the first president of the People's Republic of China,” “the year that the People's Republic of China founded,” and “the name of the only female emperor in Chinese history.” In the verbal fluency subtest, letter fluency was replaced with asking the participants to say as many words with one simple Chinese character in 1 min as possible. In the repetition part of the language subtest, all of the words and sentences were replaced with Chinese words and phrases. In the comprehension part of the language subtest, the instruction “point to the one that is a marsupial” was replaced with “point to the one that is a reptile” (the correct answer is “crocodile”), since the Chinese translations of marsupial and kangaroo are quite similar in pronunciation. The reading part of the language subtest, which contains English words that do not obey the phonic rules, was adapted by selecting five Chinese “exceptional” characters that are not pictophonetic.

The present study was designed to distinguish MCI from normal controls (NCs), and distinguish mild dementia from nondemented (MCI+NC) groups. The accuracy requirements in the present study were a type II error rate (false negatives) of ≤0.2 and a type I error rate (false positives) of ≤0.05. A null-hypothesis area under the receiver operating characteristic (ROC) curve (AUC) of 0.5 indicates no discriminating power. Previous studies1318 found that the AUCs of the Chinese versions of MMSE and MoCA were 0.71 and 0.72, respectively, for MCI, while that of ACE-III was 0.95 for dementia. Thus, the input AUC (the minimum acceptable AUC) was set to be 0.70 in the present study. When analyzing ROC curves for distinguishing MCI from NC, the ratio of sample sizes in MCI/NC groups was 1, and the required sample size of each group was ≥31. When analyzing ROC curves between mild dementia and nondemented (MCI+NC) groups, the ratio of sample sizes in nondemented/demented groups was 2, so the minimum sample sizes of the demented and nondemented groups were 24 and 48, respectively. An overall sample size of 180 with n=60 in each group would give a type I error rate of 0.025 and type II error rate of 0.05. The sample-size calculator used is available at http://www.medcalc.org/download/medcalcsetup.exe.

All statistical analyses were carried out using SPSS (version 19.0, IBM Corp., Armonk, NY, USA). A p value of <0.05 was considered statistically significant. Continuous variables are presented as mean±SD values, while enumeration data are presented as number (%) values. One-way ANOVA tests were used to compare continuous variables (age, education level, and scores on cognitive tests) among the groups. Chi-square tests were applied to categorical data. Cronbach's α coefficient was used to evaluate the internal consistency reliability of ACE-III. Convergent validity was calculated using two-tailed Pearson's correlations between the total ACE-III scores and scores on the other scales.

ROC curves for detecting MCI and mild dementia were plotted, and the AUCs were calculated. ROC curves were also combined with logistic regression analyses to calculate AUCs with adjustment for age and/or education level. The cutoff scores with the best sensitivity and specificity were determined according to the maximum Youden index (Youden index=sensitivity+specificity-1). Z tests were used for comparing two AUCs. A p value of <0.05 indicated that the two cognitive tests exhibited a significant difference in diagnostic accuracy. Sensitivity and specificity were used to provide additional information when applicable.

This study enrolled 176 participants (age 74.14±6.68 years; 55.7% men), comprising 55 NCs, 64 MCI patients, and 57 mild dementia patients. The etiologies of dementia included 27 patients with AD, 14 with VD, 10 with mixed-type dementia (AD+VD), 3 with LBD, 2 with FTD, and 1 with PDD. The baseline characteristics of the three groups are listed in Table 1. There were significant differences among the three groups in all variables except sex.

The NC group, which served as normative data, included 22 women and 33 men aged 72.2±6.48 years with an education level of 12.49±4.25 years. The ACE-III scores of the NC group were entered into the multiple regression model together with the possible confounding variables of age, education level, and sex. Age (r=−0.29, p=0.01) and education level (r=0.61, p<0.001) significantly affected the ACE-III scores, suggesting that these two parameters should be adjusted for when calculating AUCs. Sex did not significantly affect the ACE-III scores (p>0.05).

ACE-III exhibited good internal consistency reliability, with an overall Cronbach's α coefficient of 0.824. The Cronbach's α coefficients for the five subtests were 0.788 for attention, 0.744 for memory, 0.772 for fluency, 0.702 for language, and 0.740 for visuospatial abilities, indicating that every ACE-III item was needed. Two-tailed Pearson's correlation tests showed that the total ACE-III scores were negatively correlated with CDR scores (rs=−0.803, p<0.01). Meanwhile, ACE-III scores were positively correlated with MMSE (rs=0.729, p<0.01) and MoCA (rs=0.830, two-tailed p<0.01) scores.

The AUC of ACE-III was 0.88 for diagnosing MCI, which was significantly higher than those of MMSE (AUC=0.72, p<0.05) and MoCA (AUC=0.76, p<0.05). At the optimal cutoff score of 88/89, ACE-III yielded an acceptable sensitivity (0.75) and a satisfactory specificity (0.89), which were higher than the sensitivity and selectivity of MMSE (0.64 and 0.63, respectively, at the optimal cutoff score of 28/29) and MoCA (0.67 and 0.77 at the optimal cutoff score of 24/25).

The AUCs of ACE-III, MMSE, and MoCA were 0.95, 0.95, and 0.91, respectively, for mild dementia (p>0.05). An ACE-III cutoff score of 74/75 resulted in a sensitivity of 0.94 and specificity of 0.83. Both MMSE and MoCA at cutoff scores of 25/26 and 21/22, respectively, showed lower sensitivities (0.89 and 0.71, respectively) and higher specificities (0.88 and 0.93). All of the AUCs were calculated with adjustments for age and education level, and no significant differences were identified between each pair of adjusted and unadjusted AUCs. The ROC curves are shown in Fig. 1, and the results are listed in Table 2.

Since education level was shown to exert moderate influences (r=0.61, p<0.001) on ACE-III scores, participants were stratified into those who had ≥12 years of education (34 NCs, 34 MCI patients, and 26 dementia patients) and those who had <12 years of education (21 NCs, 32 MCI patients, and 31 dementia patients). ROC analyses were performed separately between these two groups. However, since the number of NC participants with <12 years of education was insufficient (≥31), ROC analysis was not performed for the distinction between MCI patients and NCs with <12 years of education.

As indicated in Table 3, the AUC of ACE-III was 0.90 for MCI in participants with ≥12 years of education, which was significantly larger than that of MMSE (0.68, p<0.05) and comparable to that of MoCA (0.81, p>0.05). At a cutoff score of 89/90, the sensitivity and specificity of ACE-III were 0.82 and 0.85. Compared to ACE-III, both MMSE and MoCA at their optimal cutoff scores of 28/29 and 24/25, respectively, showed lower sensitivity (0.68 and 0.77, respectively) and specificity (0.59 and 0.76).

The AUC of ACE-III was 0.97 when detecting mild dementia in participants with ≥12 years of education, which was significantly larger than that of MMSE (0.90, p<0.05) and comparable to that of MoCA (0.93, p>0.05). At a cutoff score of 82/83, ACE-III yielded satisfactory sensitivity (0.88) and specificity (0.92). The sensitivity was comparable to thos- of MMSE (0.90) and MoCA (0.88), while the specificity was higher than those of MMSE (0.73) and MoCA (0.75).

The AUC of ACE-III was 0.93 for detecting mild dementia in participants with <12 years of education, which was comparable to those of MMSE (0.98, p>0.05) and MoCA (0.90, p>0.05). A cutoff score of 70/71 yielded high sensitivity (0.94) and specificity (0.84). The sensitivity and specificity were 0.88 and 1.00, respectively, for MMSE at a cutoff score of 25/26, and 0.88 and 0.77 for MoCA at a cutoff score of 18/19.

As indicated in Table 2, ACE-III produced a significantly larger AUC than did MMSE and MoCA in detecting MCI. Along with the higher sensitivity and specificity, our results suggest that ACE-III is a valid alternative to MMSE and MoCA for detecting MCI. The original validation study of ACE-III did not investigate MCI,14 nor did the study of Wang et al.18

Our findings when comparing ACE-III and MMSE in diagnosing MCI concurred with two validation studies of ACE-R finding its obvious advantages over MMSE.2324 However, the Spanish version of ACE-III has been shown to have comparable diagnostic accuracy to MMSE (AUC=0.82 vs. 0.78) when detecting amnestic MCI.25 The different study designs might contribute to these inconsistent results since the present study targeted MCI rather than amnestic MCI. Nevertheless, ACE-III assesses a broader range of cognitive abilities with greater difficulties than does MMSE, especially on memory, language, executive, and visuospatial function. Also, unlike MMSE, ACE-III contains components measuring verbal fluency, which is associated with frontal lobe function. These features might enable ACE-III to better discriminate MCI from NCs and exert fewer ceiling effects than MMSE.

Few studies have compared the efficacies of ACE-III and MoCA in detecting MCI, although both tests are considered to be more sensitive than MMSE.91115161718 The Portuguese version of ACE-III has been validated to have a comparable AUC to MoCA (AUCs 0.82 vs. 0.78) in detecting MCI,19 while the present study showed that the Chinese version of ACE-III had a significantly larger AUC. Besides the differences in the samples and scales used between these two studies, another reason for the inconsistent results might be that the Chinese version of MoCA is not much better than the Chinese version of MMSE. One previous study found close AUCs of MoCA and MMSE for MCI (0.72 vs. 0.71),13 similarly as did the present study (0.76 and 0.72). This phenomenon is thought to be due to inadequacies in translating the original English version of MoCA to the Chinese version, resulting in prominent linguistic and cultural differences between these two versions.

In summary, these results support our finding that ACE-III was significantly superior to MMSE and MoCA when diagnosing MCI in a Chinese-speaking population.

As indicated in Table 2, ACE-III yielded a satisfactory AUC of 0.95 for mild dementia, which was comparable to those of MMSE and MoCA. Two previous studies have compared the efficacies of ACE-III and MMSE in screening dementia. The AUC of the Spanish version of ACE-III was found to be similar to that of MMSE (0.92 vs. 0.91),16 while Wang et al.18 found that ACE-III was significantly superior to MMSE (0.95 vs. 0.83). Therefore, the AUCs of ACE-III were similar among the two previous studies and the present one, while they were quite different for MMSE. This suggests that ACE-III has a satisfactory diagnostic accuracy across different samples, while MMSE is influenced more by sample disparities. In addition, two previous studies found that ACE-III does not have significant benefit over MoCA in detecting dementia, which is consistent with the present results.2627 Overall, ACE-III revealed excellent diagnostic accuracy in diagnosing mild dementia, comparable to MMSE and MoCA.

As indicated in Table 3, ACE-III was significantly superior to MMSE when screening for both MCI and mild dementia in higher-educated participants. However, when detecting dementia in lower-educated participants, ACE-III was not better than MMSE. Compared with MMSE, ACE-III has been designed with more-comprehensive domains and presents more-challenging tasks for participants, which might result in its better ability to detect cognitive decline in higher-educated individuals. Consistently, ACE-R is also found to be prior to MMSE in higher-educated patients, but not in lower-educated patients.28 The present study is the first to demonstrate this feature for ACE-III. In addition, these results suggest that ACE-III rather than MMSE should be applied to higher-educated individuals. On the other hand, MMSE might be preferable when screening for dementia in individuals with <12 years of education, since MMSE is easier to complete and less time-consuming to administer.

Our study found that the optimal cutoff score for dementia was 74/75, which is lower than that of 83/84 found by Wang et al.18 One possible reason for this difference is sample disparities. Our results showed older age and lower education levels were associated with lower ACE-III scores, similarly as did the previous studies.1516171820 Since the dementia group in our study had a lower education level (9.68±4.27 vs. 11.3±1.4 years) and were older (74.89±6.90 vs. 71.9±7.8 years) than the subjects in the study of Wang et al.,18 it was reasonable that the optimal cutoff score was lower in the present study. The SD for the education level was also higher in our sample (4.27 vs. 1.4). This might further decrease the mean scores of ACE-III, although it was more representative for the sampled population. Our results revealed that the optimal cutoff score was 82/83 for detecting dementia in higher-educated participants, which was very close to that found (83/84) by Wang et al.18

Our findings must be interpreted in the light of certain limitations. Firstly, most of the demented subjects were diagnosed with AD, mixed-type dementia, or VD, and hence it is possible that recruitment bias was present. Secondly, the present study was carried out in a tertiary hospital with single-center design, which might also have introduced bias. Finally, the sample was too small to carry out qualified ROC analyses for detecting MCI in lower-educated participants, resulting in the lack of adjusted data under that condition. However, since ACE-III showed satisfactory diagnostic accuracy for MCI—which was significantly superior to those of MMSE and MoCA (Table 2)—it appears reasonable to choose ACE-III for detecting MCI in lower-educated participants.

In conclusion, the present study has verified that the Chinese version of ACE-III is an effective and accurate tool for screening MCI and mild dementia. ACE-III was found to be significantly superior to MMSE and MoCA for detecting MCI. In higher-educated individuals, ACE-III yielded significant advantages over MMSE when screening for both MCI and mild dementia. Further studies are necessary to validate the Chinese version of ACE-III in larger samples and across various subtypes of dementia and MCI.