1. Background
In near future, with increasing the number of aged people in the world, dementia would become a major public health problem, in most countries especially the developing ones (1). In Iran, as well as many other developing countries, life expectancy has improved in recent decades (2) and the proportion of people 60 years old and over hasincreased from 5.5% in 1975 to 8.5% in 2016 (3, 4). It is estimated that by 2050, the population of aged people in Iran will reach 27.3% (2) and the country will face great challenges meeting age-specific care needs of this group. Considering the important role of early dementia detection and intervention in increasing the quality of life of people with dementia and their care-givers and lowering their suffering (5), the provision of simple, short, culturally appropriate, and inexpensive cognitive screening tools appears to be a necessity. The clock drawing test (CDT) is a simple, quick, and inexpensive test that was first used by Shulman et al. for screening of cognitive problems (6). The CDT is well accepted by patients and health care professionals and therefore seems to be an appropriate cognitive screening tool in geriatric primary care settings (6, 7). It has been evaluated in several studies in the last decades, with more than 15 highly validated interpretative scoring systems (6, 8). CDT as well as mini mental state examination (MMSE) (9) can be applied to assess several neuropsychiatric functions. CDT has been adopted, validated and used successfully in the context of different languages and cultures (10-14). Aprahamian et al. concluded that despite using various scoring systems, CDT results are highly correlated with other valid cognitive screening tools and can be used as a single screening test or as a part of a larger assessment protocol (7). It can also be used as a longitudinal measure of cognition to evaluate cognitive changes over time (15).
In Iran, alongside the effort of the authors of this article (16), Ehtezazi et al. (17) used a real clock, a metal circle, and magnetic numbers to validate different scoring methods of CDT. They reported that CDT is a valid cognitive screening tool and the highest discriminatory power belonged to reading, clock setting, completion, and drawing activities, respectively. However, it seems difficult and time-consuming to utilize their proposed instrument in primary care units. Mainland et al. stated that simpler CDT scoring methods could be as sufficient as more detailed ones in screening dementia (15). Therefore, the present authors selected Shulman’s modified scoring method (18) which is easy to use and has shown the best accuracy level among the different scoring methods in many comparative studies (14, 19).This method has already been validated in many countries such as England (20), Brazil (21) and Turkey (13).
2. Objectives
Thus, the goal of this study was to assess the psychometric properties of the CDT using Shulman’s modified scoring method.
3. Materials and Methods
This psychometric study was conducted in Iran’s Alzheimer Association from May to September 2012. Using convenient sampling method, the participants were recruited voluntarily from patients, care givers, and other individuals, all 60 years old or over, who were referred to the clinics. The staff in charge of registration was unaware (blind) of the aim of the study and the participants’ diagnosis. The inclusion criteria were (1) aged 60 or above, and (2) having at least 4 years of formal education. Patients with advanced dementia who could not communicate, and those who had visual or auditory problems or depression were excluded. A validated short form (11 items) of geriatric depression scale (GDS) (22) was used to detect depressed participants; GDS score ≥ 6 was used as a cut of point for depression and therefore for exclusion.
This study was approved by the Ethics Committee of the University of Social Welfare and Rehabilitation Sciences and the ethical approval code is IR.USWR.REC.1392.128. Registered patients were free to participate or leave the study and an informed consent was taken from the participants and also from the legal guardians of the demented group. The participants were ensured about the privacy of their information. Each participant received a questionnaire, consisting of demographic variables, GDS, MMSE, and two big pre-drawn circles shaped like an old clock. Trained raters administered the questionnaires. CDT was implemented by two trained raters in the same visit. Each rater put the pre-drawn clock shaped circle in front of the participant and asked him or her to complete the numbers of the clock. After finishing the numbers, raters asked the participant to set the clock at 11: 10. Thereafter, the raters scored the results by Shulman’s six-point scoring method (6, 18) in which a perfect clock gets 5 and no clock shape gets zero. To assess test-retest reliability, the CDT was performed on 20 of the participants again after one week (negligible deterioration in cognitive level was expected during this period). Thereafter, the researchers searched for the participant’s medical diagnosis in their records (all registered diagnoses in Iranian Alzheimer Association are based on the DSM-IV-TR criteria) and all participants were divided into two groups of: Participants with and without dementia.
Statistical analysis was performed using the 20th version of SPSS . Intra class correlation coefficient (ICC) and Cohen’s Kappa coefficient (κ) were used to assess inter-rater and test-retest reliability, respectively. Concurrent validity was evaluated by calculating Spearman’s rank correlation coefficient between MMSE and the CDT scores. Mann-Whitney U test, because of ordinal variables and the lack of normal distribution, was used to assess the mean differences between scores of the dementia and non dementia groups. The normality of the scores was assessed using Kolmogorov Smirnov test. receiver operative characteristics (ROC) curve was drawn and the area under the ROC curve, the cut-off point, sensitivity, and specificity were calculated and analyzed.
4. Results
The sample size was 74, with a mean age of 72.2 ± 7.95 years. A total of 30 (41%) participants were diagnosed with dementia. Table 1 presents the demographic characteristic of the participants.
Table 1.Demographic Charactristics, Dementia Diagnosis Status and MMSE Scores of the Participant
| Characteristics | Valuesa |
|---|---|
| Age, y | |
| 60 - 69 | 32 (43.2) |
| 70 - 79 | 27 (36.5) |
| ≥ 80 | 15 (20.3) |
| Sex | |
| Female | 37 (50) |
| Male | 37 (50) |
| Educational level | |
| Elementary school (4 - 5 y) | 10 (13) |
| Primary high school (6 - 8 y) | 18 (24) |
| Secondary high school (8 - 12 y) | 22 (29) |
| University graduated | 24 (32) |
| Dementia diagnosis | |
| Yes | 30 (40.5) |
| No | 44 (59.5) |
| MMSE scoresb | |
| Cognitive problem (≤ 20) | 27 (36.5) |
| Susceptible (21 - 22) | 2 (2.7) |
| Normal cognition (≥ 23) | 45 (60.8) |
a Vlues are expressed as No. (%).
b MMSE scores groups were selected according to the validation study in Iran (23).
4.1. Validity of CDT
The validity of CDT was evaluated in two ways: (1) Concurrent validity, using the correlation between the MMSE and the CDT scores by Spearman’s rank correlation coefficient (r = 0.782, P < 0.001), and (2) differential validity, using the mean CDT score difference between participants with and without dementia, showing a significant difference at P < 0.001 (1.63 in the dementia group compared to 4.00 in the non-dementia group).
4.2. Reliability
Among all 74 participants, The ICC for inter-rater reliability was 0.964, statistically significant at P < 0.001. Re-tests were conducted among 20 of the participants and Kappa statistics for test-retest reliability was 0.554 (P < 0.001).
4.3. Sensitivity, Specificity, and Cut-Off Point
The area under the ROC curve (Figure 1) was 0.789 (statistically significant at P < 0.001). The best cut-off point for the participants was 3/4 with 90% sensitivity and 73% specificity. The positive and negative predictive values were 69% and 91%, respectively.
Figure 1.
ROC curve for the CDT
5. Discussion
The results showed that Shulman’s simple scoring method of the clock drawing test has acceptable psychometric properties among the aged in Iran. It was also a well accepted instrument among the participants, as it has been shown in previous studies (15). In the present study, the inter-rater reliability of CDT was excellent (ICC = 0.96) which is comparable to 0.96, 0.89 and 0.86 in the studies of Fuzikawa et al. (11), Schramm et al. (24) and Can et al. (13) respectively.
The significant difference between mean CDT scores of the dementia versus non-dementia groups showed an acceptable differential validity. It implies that CDT is capable of discriminating between patients with dementia and without dementia in the populations.
The correlation of CDT and MMSE scores (0.782) indicated a good concurrent validity, consistent with other studies (0.80) and (0.73) (18, 25). Interestingly, Schramm et al. (24) and Fuzikawa et al. (26) reported a moderate (0.68 and 0.64, respectively), and Cecato et al. (21) reported a weak correlation between CDT and MMSE.
The present study showed that, at the cut-off point of 3/4, the sensitivity and specificity of CDT were 90% and 73%, respectively. This is similar to findings of Aprahamian et al. (25) that reported a sensitivity of 0.85 for CDT, using Shulman scoring system. It has been reported that for people with lower educational level (1 - 5 years), the sensitivity and specificity of CDT were adequate and high, whereas for the group with higher education level (> 5 years), there were lower specificity and sensitivity values (25). This puts forward the issue of the impact of education on subjects’ ability to perform CDT. Similarly, Cecato et al. (21) suggested that for elder individuals with high education levels, CDT did not seem to be an appropriate test for detecting cognitive impairments. Interestingly, Fuzikawa et al (26) reported that CDT can act as a proper test in developing countries, where the number of literate persons is normally low. However in recent times, this view has been seriously challenged; for example, Kim et al. (27) concluded that the CDT performance in old people with less than 6 years education level should be interpreted cautiously. Also, it has been reported that education may strongly affect CDT performance among non-demented old people in Asia and South America, with high numbers of illiterate people (28-30). Therefore, our sample with almost 85% of participants having more than 5 years education can be considered as a proper sample, which is less susceptible to less systematic bias in analysis.
In this study the best cut-off point for Shulman’s modified scoring method was 3/4, showing that scores of 0 to 3 are abnormal while scores of 4 and 5 are normal. This cut-off point, was similar to Shulman’s suggested cut off point, and had 90% sensitivity and 73% specificity among our participants. The same cut-off point score recorded 81% and 93% sensitivity and 79% and 55% specificity in studies of Schramm et al. (24) and Storey et al. (14), respectively. However, sensitivity and specificity of this score among old people with low educational level was lower, as in Lourenco’s study (31) (65% and 57%, respectively). This point re-emphsizes the relative importance of education in cognitive evaluation.
The significant area under the ROC curve (0.789) showed good discriminatory potential of the clock drawing test, that is comparable with 0.79% in Storey et al. (14), 0.82 in Can et al. (13) and 0.68 in Lourenco et al. (31) studies.
5.1. Limitations
First, as we needed a sample that can read and write, so be able to respond to all items of MMSE and complete the test (23) the results of this study cannot be generalized to illiterate population. The other limitation is the small sample size. Alzheimer and other related dementias are relatively unknown in the Iranian population and patients are diagnosed with significant delay as a result of late referral to medical services. Therefore, the majority of patients referred to the Association are severely impaired and almost un-testable. These two limitations, illiteracy and severity of dementia, limited the number of eligible participants from the referred to the Association clinic over the 5-month study period. Also, authors did not compare the properties of the different scoring methods as seen in several studies (13, 14, 21, 24), which could provide a more accurate and precise scoring system for differentiating cognitively impaired from unimpaired aged people.
In conclusion, CDT was found to be an appropriate dementia screening tool among literate old people of Tehran. As a developing country, Iran has recently been faced with the challenges associated with an increasing number of old people. Considering the limited financial and human resources available, a simple, quick, well-accepted, and cost-benefit tool for cognitive screening can be very useful in this situation. Consequently, CDT can be used as a valid and reliable screening instrument for cognitive impairments in geriatric primary care settings. Further studies are required to evaluate the applicability of CDT among other groups with different characteristics and to evaluate other types of construct validity with bigger sample size.
