Assessing Negative Automatic Thoughts: Psychometric Properties of the Turkish Version of the Cognition Checklist

authors:

avatar Sedat Batmaz 1 , * , avatar Ozgur Ahmet Yuncu 2 , avatar Sibel Kocbiyik 3

Department of Psychiatry, School of Medicine, Gaziosmanpasa University, Tokat, Turkey
Psychiatry Clinic, Ankara Training and Research Hospital, Ankara, Turkey
Psychiatry Clinic, Ataturk Training and Reseach Hospital, Ankara, Turkey
Iranian Journal of Psychiatry and Behavioral Sciences: Vol.9, issue 4; e3444
published online: December 23, 2015
article type: Original Article
received: July 6, 2015
accepted: September 1, 2015
DOI: https://doi.org/10.17795/ijpbs-3444

how to cite: Batmaz S, Ahmet Yuncu O, Kocbiyik S. Assessing Negative Automatic Thoughts: Psychometric Properties of the Turkish Version of the Cognition Checklist. Iran J Psychiatry Behav Sci. 2015;9(4):e3444. https://doi.org/10.17795/ijpbs-3444.

Abstract

Background:

Beck’s theory of emotional disorder suggests that negative automatic thoughts (NATs) and the underlying schemata affect one’s way of interpreting situations and result in maladaptive coping strategies. Depending on their content and meaning, NATs are associated with specific emotions, and since they are usually quite brief, patients are often more aware of the emotion they feel. This relationship between cognition and emotion, therefore, is thought to form the background of the cognitive content specificity hypothesis. Researchers focusing on this hypothesis have suggested that instruments like the cognition checklist (CCL) might be an alternative to make a diagnostic distinction between depression and anxiety.

Objectives:

The aim of the present study was to assess the psychometric properties of the Turkish version of the CCL in a psychiatric outpatient sample.

Patients and Methods:

A total of 425 psychiatric outpatients 18 years of age and older were recruited. After a structured diagnostic interview, the participants completed the hospital anxiety depression scale (HADS), the automatic thoughts questionnaire (ATQ), and the CCL. An exploratory factor analysis was performed, followed by an oblique rotation. The internal consistency, test-retest reliability, and concurrent and discriminant validity analyses were undertaken.

Results:

The internal consistency of the CCL was excellent (Cronbach’s α = 0.95). The test-retest correlation coefficients were satisfactory (r = 0.80, P < 0.001 for CCL-D, and r = 0.79, P < 0.001 for CCL-A). The exploratory factor analysis revealed that a two-factor solution best fit the data. This bidimensional factor structure explained 51.27 % of the variance of the scale. The first factor consisted of items related to anxious cognitions, and the second factor of depressive cognitions. The CCL subscales significantly correlated with the ATQ (rs 0.44 for the CCL-D, and 0.32 for the CCL-A) as well as the other measures of mood severity (all Ps < 0.01). To a great extent, all items of the CCL were able to distinguish the clinical and non-clinical groups, suggesting the scale has high discriminating validity.

Conclusions:

The current study has provided evidence that the Turkish version of the CCL is a reliable and valid instrument to assess NATs in a clinical outpatient sample.

1. Background

The cognitive theory of emotional disorders developed by Beck (1) has been enormously influential in psychiatry (2-4). Hypotheses about the relationship between cognition and emotion that were derived from this theory have led to a greater understanding of many psychopathological states, and to an effective treatment modality, i.e., cognitive behavioral therapy, which has impressively shaped the psychiatry literature ever since (5-8). Beck’s cognitive theory posits that mood states may be discriminated on the basis of their unique cognitive contents (1, 9). According to the cognitive content specificity hypothesis, automatic thoughts and subjectively perceived emotional states should be positively related (10-12).

Beck’s theory of emotional disorder suggests that negative automatic thoughts (NATs) and the underlying schemata affect one’s way of interpreting situations and result in maladaptive coping strategies (1, 13). This theory proposes a three-layer cognitive structure, where NATs are topographically located on the outermost surface. NATs are the most easily accessible cognitions, and they tend to be the easiest to work on with patients during therapy sessions. NATs distort reality, are emotionally distressing, and interfere with patients’ functionality. Depending on their content and meaning, NATs are associated with specific emotions, and since they are usually quite brief, patients are often more aware of the emotions they feel (13). This relationship between cognition and emotion, therefore, is thought to form the background of the cognitive content specificity hypothesis.

Before cognitive variables were described in a taxonomy by Kendall and Ingram (14), there was confusion about the use of the term cognitive content specificity. Yet, now the term is more widely accepted as “a more specific claim that certain themes of semantic content in self-reported automatic thoughts are unique to either depression or anxiety” (15). Although other theories have attempted to explain the cognitive content specificity, e.g., the self-discrepancy theory by Higgins (16), the most investigated formulation to date has been the hypothesis by Beck (9). After the development of an assessment tool to discriminate between depressive and anxious cognitions, i.e., the cognition checklist (CCL) (10), research aiming to improve the differential diagnosis of mood and anxiety disorders has increased substantially (12, 15, 17-25). Researchers focusing on cognitive content specificity have suggested that instruments like the CCL might be an alternative to make a diagnostic distinction between depression and anxiety. This suggestion stemmed from clinical observations demonstrating that depressive cognitive content was focused on themes related to negative self-evaluation, hopelessness, and pessimism about the future (9, 26, 27), whereas anxious cognitive content was more focused on physical or psychological threat, and an inability to cope with danger (26, 28). Another way of differentiating depressive and anxious cognitive content was proposed to be associated with the temporal focus of cognitions, where depressive cognitions were more likely to be past-oriented, and anxious cognitions future-oriented (29).

A meta-analysis on the cognitive content specificity hypothesis concluded that the hypothesis was only supported for depressive cognitive content (15). The authors argued that anxious cognitive content’s poor performance in demonstrating specificity might be due to the possibility that (i) anxious cognitive content might not be specific to anxiety and (ii) the themes involved in anxious cognitive content might be shared variables between depression and anxiety. This is further supported by studies using prototypical cognitions to distinguish different diagnostic categories (18, 30) and on positive and negative affectivity (24, 26, 31, 32).

Currently available measures to assess negative cognitions in depression or anxiety for adults are limited. In addition to the CCL (10), the literature review provided us with the following measures: (i) the automatic thoughts questionnaire (ATQ) (33), (ii) the Crandell cognitions inventory (CCI) (34), (iii) the anxious self-statements questionnaire (ASSQ) (35), (iv) the UBC cognitions inventory (UBC-CI) (30), (v) the agoraphobic cognitions questionnaire (ACQ) (36), and (vi) the body sensations questionnaire (BSQ) (36). These questionnaires, except for the ATQ, the CCI, and the depression subscale in the UBC-CI, focus on thematically related cognitions to anxiety, whereas the former questionnaires focus solely on depressive cognitions. Apart from the UBC-CI, which has both subscales for depression and anxiety, the CCL is, therefore, unique in that it consists of two different subscales focused on depressive or anxious cognitions. The anxiety subscale (CCL-A) has less prototypical items in the sense that they do not particularly represent core features of the aspects of specific anxiety disorders. Therefore, these items may be classified as general in terms of anxiety related cognitions, i.e., future-oriented threat. The UBC-IC, however, consists of more prototypical and disorder-specific anxiety subscales, e.g., worry, panic, somatic preoccupation, and social fear. This distinction might position the CCL-A as a more transdiagnostic perspective on anxiety, whereas the specific anxiety subscales of the UBC-IC might be more relevant for research involving disorder-specific approaches to anxiety. The depression subscale of the CCL (CCL-D), as reported in the literature (15), may also be conceptualized as disorder specific in the sense that its items reflect the core aspects of depression.

The CCL was initially developed to differentiate anxiety and depression and to measure the frequency of automatic thoughts (10). It was initially thought to explicitly test the cognitive content specificity hypothesis of the cognitive model (1, 27). Its psychometric properties indicate that it is a reliable and valid tool. Cronbach’s α values range from 0.90 - 0.91 to 0.92 - 0.93 in psychiatric outpatients and 0.86 and 0.90 in students, for the CCL-A and CCL-D, respectively. It has also been shown to have high test-retest reliability and concurrent validity with scales measuring depression and anxiety severity (10, 37). The subscales have also demonstrated evidence for discriminant validity, differentiating patients diagnosed with depression or anxiety (10, 37). The results have also indicated that the CCL-D and CCL-A are moderately correlated with each other, which might be due to the shared variance between the subscales (37). Factor analytic studies have generally revealed that the CCL consists of two subscales, which correspond to depression and anxiety related negative cognitions (10, 37, 38), although some findings differ (19). Apart from these initial studies, later research has also consistently reported that the CCL is a reliable and valid tool for research purposes (39-41).

2. Objectives

The aim of the present study was to assess the psychometric properties of the Turkish version of the CCL in a psychiatric outpatient sample. The hypotheses were that the CCL subscales would correlate significantly with measures of depressive and anxiety symptoms and a similar scale measuring NATs and the CCL subscales could distinguish between clinical diagnostic subgroups.

3. Patients and Methods

3.1. Participants

A total of 425 psychiatric outpatients 18 years of age and older presenting to one secondary and two tertiary healthcare services in two different cities were recruited. Participants were excluded from the study if they (i) were diagnosed with psychotic disorders, bipolar mood disorders, organic mental disorders, substance use disorder, dementia, and/or mental retardation, (ii) suffered from a medical/neurologic disorder not currently under control, (iii) were suicidal at the time of the intake interview, and (iv) had a history of head trauma, brain surgery, or electroconvulsive therapy.

3.2. Assessment Instruments

Demographic and Clinical Data Form: This form was developed by the researchers and the demographic data, i.e., age, gender, level of education, marital status, occupation status, and clinical variables, i.e., primary psychiatric diagnosis, family history of mental disorder, comorbid medical disorder, were recorded onto it.

Mini International Neuropsychiatric Interview (MINI) (42): The MINI is a structured clinical diagnostic interview for mental disorders. In the present study, all participants were diagnosed by experienced clinicians according to the Diagnostic and Statistical Manual of Mental Disorders version IV (DSM-IV) (43) with the Turkish version of the MINI (44).

Cognition checklist (CCL) (10): The CCL is a 26-item 5-point self-report scale that is used to assess the frequency of NATs reported by patients (0 = never, 4 = always), and it has two subscales, a 14-item depression subscale (CCL-D), characterized by hopelessness, generally pessimistic assessments of the world and the future, and negative self-judgment, and a 12-item anxiety subscale (CCL-A), characterized by general future-oriented physical or psychological threat, and danger (15, 30). The respondents are required to rate the frequency of their depressive and anxious automatic thoughts across a variety of situations, e.g., while feeling pain or physical discomfort, attending a social situation, working on a project, with a friend (possible ranges 0 - 56 for CCL-D, and 0 - 48 for CCL-A). For the present study, the respondents were asked to rate their thoughts regardless of the situation. For the translation of the CCL, guidelines widely used in cross-cultural research were followed (45, 46). First, the developer of the scale was contacted by e-mail, and after his approval, the scale was translated into Turkish by the first author of this article. The translated scale was independently back-translated by two bilingual experts in the field, and all translations were compared with the original scale. After reviewing the original and translated versions, a final version of consensus was adopted.

Automatic thoughts questionnaire-negative (ATQ) (33): The ATQ is a 30-item 5-point self-report scale that assesses the frequency of NATs. For each item, respondents are asked to indicate how frequently each thought occurred during the past week (1 = not at all, 5 = all the time). The ATQ predominantly concentrates on negative cognitions characteristic of depression, e.g., loss, failure, self-depreciation (10) and consists of statements that reflect different aspects of depression, e.g., demoralization, self-criticism, brooding, amotivation, and interpersonal disappointment (47). The ATQ has been reported to have excellent psychometric properties and to differentiate between depressed and non-depressed groups (48, 49). The Turkish version of the ATQ, which has good reliability (Cronbach’s α = 0.93) and validity, was used (50). Only the total score of the ATQ was used in the analyses (possible range = 30 - 150).

Hospital anxiety and depression scale (HADS) (51): The HADS consists of 14 items, divided into two equal subscales of depression and anxiety. It is a 4-point self-report instrument, and cut-off scores of 7 for depression, and 10 for anxiety have been proposed. The Turkish version of the HADS (used in the present study) is a reliable (Cronbach’s α = 0.78 for depression, and 0.85 for anxiety) and valid instrument (52). Both subscale scores were calculated for analyses.

3.3. Procedure

The diagnostic interview was administered face-to-face at intake by trained psychiatrists. Participants completed the self-report measures after the intake interview. All questionnaires were administered in random order. It took about 20 - 30 minutes for all the questionnaires to be completed. No compensation of any sort was offered. All participants signed a written informed consent before the study, and the respective local ethics committees approved the study design.

3.4. Statistical Analyses

All analyses were performed using IBM SPSS for Windows, Version 22.0 (53). Participants’ demographic and clinical data were analyzed by descriptive statistics. For group comparisons, Student’s t-test was applied. An exploratory factor analysis (principal axis factoring) was performed, followed by an oblique rotation (direct oblimin). Factors for extraction were selected by examining eigenvalues (54), and the scree plot, and by conducting a parallel analysis (55-58). To assess the internal consistency of the scale, Cronbach’s α was computed. The correlation coefficient was calculated for test-retest reliability over a 4-week period. For concurrent validity, bivariate Pearson correlation analyses were conducted with the measures of depression and anxiety severity, and of NATs. The correlation coefficients were compared with each other for statistical significance by Steiger’s Z test (59). For discriminant validity, partial correlation coefficients between the same- and opposite-affect rating scales and the CCL subscales were calculated. Statistical significance was set at a P-value of < 0.05.

4. Results

4.1. Descriptive Statistics

A total of 425 psychiatric outpatients 18 years of age and older (60.9 % female; mean age = 37.9 years, SD = 12.76, range = 18 - 65) were recruited for the study. Almost two thirds of the participants (n = 275, 64.7 %) were married, and 31.3 % of them (n = 133) were single. Almost all the participants (n = 382, 89.9%) were at least graduates of high school, and 54.6 % (n = 232) had a job with a regular income.

The primary DSM-IV diagnoses of the participants were as follows: 32.24 % (n = 137) depressive disorders (e.g., major depressive disorder, dysthymia) and 28.94 % (n = 123) anxiety disorders (e.g., generalized anxiety disorder, panic disorder, social anxiety disorder, obsessive-compulsive disorder, posttraumatic stress disorder). Over one third of the participants had a comorbid psychiatric diagnosis, the most common being a comorbid depressive and anxiety disorder (37.41 %, n = 159). Seventy-one of the participants (16.7 %) reported that they had a family member diagnosed with some kind of mental disorder, and 119 of the participants (28 %) were also suffering from a comorbid medical condition.

The mean scores and standard deviations of the individual items and the total score of the CCL subscales as well as the other scales used in the current study are presented in Table 1.

Table 1.

Descriptive Statistics of the Individual Items and the Total Score of the Subscales of the CCL, the ATQ, and the Scales of Mood Severity

ValuesaMinimumMaximumPossible Range
HADS-Dep7.11 ± 4.800210 - 21
HADS-Anx9.55 ± 4.810210 - 21
ATQ-Total61.01 ± 25.983412030 - 150
CCL-D5.59 ± 6.030240 - 56
CCL-A6.97 ± 8.130340 - 48
Item 10.50 ± 0.66040 - 4
Item 20.36 ± 0.57040 - 4
Item 30.52 ± 0.77040 - 4
Item 40.96 ± 1.01040 - 4
Item 50.26 ± 0.51040 - 4
Item 60.24 ± 0.49040 - 4
Item 70.38 ± 0.57040 - 4
Item 80.29 ± 0.60040 - 4
Item 90.25 ± 0.51040 - 4
Item 100.29 ± 0.60040 - 4
Item 110.25 ± 0.50040 - 4
Item 120.26 ± 0.62040 - 4
Item 130.38 ± 0.76040 - 4
Item 140.65 ± 0.86040 - 4
Item 150.79 ± 0.92040 - 4
Item 160.43 ± 0.75040 - 4
Item 170.43 ± 0.71040 - 4
Item 180.43 ± 0.74040 - 4
Item 190.72 ± 0.97040 - 4
Item 200.60 ± 0.98040 - 4
Item 210.64 ± 0.96040 - 4
Item 220.51 ± 0.77040 - 4
Item 230.48 ± 0.86040 - 4
Item 240.62 ± 0.92040 - 4
Item 250.89 ± 1.10040 - 4
Item 260.43 ± 0.81040 - 4

aData are presented as mean ± SD.

4.2. Internal Consistency and Split-Half Reliability

The internal consistency of the CCL was excellent (Cronbach’s α = 0.95). The corrected item-total correlation (ITC) coefficients ranged from 0.41 (“I am a social failure.”) to 0.81 (“Something awful is going to happen.”). Deletion of none of the items resulted in an increase in the Cronbach’s α value of the scale. The ITC values are shown in Table 2.

Table 2.

Factor Loadings After the Exploratory Factor Analysis, and the Corrected Item-Total Correlations of the CCL

Factor 1Factor 2CommunalitiesITC
Item 10.210.320.230.47
Item 20.050.720.560.64
Item 30.010.550.310.48
Item 40.090.590.290.41
Item 50.120.630.500.62
Item 60.100.710.430.49
Item 70.040.730.570.64
Item 80.180.390.270.48
Item 90.090.800.560.56
Item 100.240.430.360.56
Item 110.150.650.550.64
Item 120.280.580.600.71
Item 130.400.490.630.75
Item 140.500.200.410.63
Item 150.670.050.490.66
Item 160.730.050.580.69
Item 170.710.090.580.69
Item 180.600.210.550.70
Item 190.660.140.570.73
Item 200.970.330.840.69
Item 210.750.060.620.74
Item 220.520.290.540.70
Item 230.620.210.580.74
Item 240.750.150.710.81
Item 250.800.110.550.63
Item 260.370.400.470.67

Abbreviation: ITC: corrected item-total correlation.

The test items were split in half, where both halves consisted of 13 items, and both halves correlated with each other (r = 0.71). The Cronbach’s α values were 0.89 for part 1 and 0.94 for part 2. The Spearman-Brown split-half reliability coefficient was good (0.83).

The internal consistencies of the CCL depression and anxiety subscales were excellent as well (Cronbach’s αs 0.90 and 0.94, respectively). The ITC coefficients ranged from 0.47 to 0.71 for the CCL-D, and from 0.63 to 0.78 for the CCL-A. Deletion of none of the items resulted in an increase in the Cronbach’s α values of the subscales.

4.3. Test-Retest Reliability

For a subgroup of patients (n = 70, 16.47 %), the test-retest correlation coefficient was calculated over a 4-week period. The results were satisfactory (r = 0.80, p < 0.001 for CCL-D, and r = 0.79, P < 0.001 for CCL-A). The test-retest reliabilities of the subscales did not differ.

4.4. Exploratory Factor Analysis

The Kaiser-Meyer-Olkin measure was 0.84, and the Bartlett’s test of sphericity was highly significant (χ2 = 10716.83, P < 0.001). An exploratory factor analysis, followed by an oblique rotation, revealed that a two-factor solution best fit the data. This bidimensional factor structure explained 51.27 % of the variance of the scale (eigenvalues 11.45, and 1.88, respectively). The first factor consisted of items related to anxious cognitions, and the second factor of depressive cognitions. Factor loadings of the items are shown in Table 2.

The factor loadings did not differ from the original scale, except for items 14 (“I have become physically unattractive.”) and 26 (“I’m losing my mind.”), which loaded on the opposite-affect factor more strongly. Items that cross-loaded on both factors (strength of loading difference on different factors < 0.30) were items 1 (“I’m worthless.”), 8 (“I’ve lost the only friends I’ve had.”), 10 (“I’m worse off than they are.”), 13 (“Nothing ever works out for me anymore.”), and 26. The only items that failed to load > 0.40 on their corresponding factors were items 1, 8, 14, and 26. Yet, all of these items were left in their original subscale, as they thematically corresponded more appropriately to their same-affect factor.

4.5. Concurrent Validity

Table 3 presents the correlations between the CCL subscales and other measures used in the study. As hypothesized, the CCL subscales significantly correlated with the ATQ (rs 0.44 for the CCL-D, and 0.32 for the CCL-A) as well as the other mood severity measures (all Ps < 0.01). As expected, the CCL-D correlated more strongly with the HADS-D (r = 0.43), and the CCL-A with the HADS-A (r = 0.42). However, these correlations with the mood severity ratings were not statistically significantly different (Zs 1.64 for the comparison of correlation coefficients between the CCL-D and the same- and opposite-mood severity scales, and 0.83 for the comparison of correlation coefficients between the CCL-A and the same- and opposite-mood severity scales). The stronger correlation between the CCL-D and the ATQ than between the CCL-A and the ATQ was also expected, as the ATQ focuses predominantly on depressive cognitions, and the former correlation was statistically significantly different from the latter (Z = 1.97, P < 0.05). These results demonstrate that there is concurrent validity of the scale.

Table 3.

Bivariate and Partial Correlations Between the CCL Subscales and the Other Measures

Correlation coefficientsHADS-DepHADS-AnxATQ
CCL-D
r.43a.33a.44a
Partial r.30b.17.37a
CCL-A
r.37a.42a.32a
Partial r.09.24b.17b

aP < 0.01.

bP < 0.05.

4.6. Discriminant Validity

Participants were divided into four groups according to the cut-off scores of the HADS subscales (depressed / non-depressed, and anxious / non-anxious), and the mean scores of the individual items and the total score of the CCL subscales were compared between these groups. Except for item 8 among the participants with anxiety, all items were able to distinguish the groups, suggesting that the scale has high discriminating validity (all Ps < 0.05). The results are shown in Table 4.

Table 4.

Discriminating Between Clinical and Non-Clinical Participants According to the CCL Scoresa

CCLHADS-DepHADS-Anx
≥ 7 (n = 217)< 7 (n = 208)t≥ 10 (n = 202)< 10 (n = 223)t
Item 10.58 ± 0.660.41 ± 0.652.632b0.58 ± 0.670.42 ± 0.652.556c
Item 20.43 ± 0.580.28 ± 0.562.794b0.42 ± 0.600.30 ± 0.542.168c
Item 30.65 ± 0.830.38 ± 0.693.643b0.63 ± 0.830.43 ± 0.712.716b
Item 41.15 ± 1.020.77 ± 0.963.882b1.09 ± 1.030.85 ± 0.972.535c
Item 50.39 ± 0.610.13 ± 0.345.464b0.35 ± 0.560.19 ± 0.453.226b
Item 60.31 ± 0.540.16 ± 0.423.305b0.32 ± 0.550.17 ± 0.423.218b
Item 70.54 ± 0.620.20 ± 0.446.524b0.51 ± 0.620.26 ± 0.494.742b
Item 80.36 ± 0.600.21 ± 0.582.738b0.34 ± 0.550.24 ± 0.631.798d
Item 90.42 ± 0.620.07 ± 0.257.620b0.38 ± 0.620.13 ± 0.345.146c
Item 100.40 ± 0.670.18 ± 0.493.801b0.41 ± 0.680.18 ± 0.504.033b
Item 110.37 ± 0.570.13 ± 0.385.063b0.35 ± 0.570.16 ± 0.403.980b
Item 120.38 ± 0.770.13 ± 0.374.342b0.37 ± 0.760.16 ± 0.443.501b
Item 130.59 ± 0.930.16 ± 0.406.130b0.57 ± 0.910.21 ± 0.525.090b
Item 140.83 ± 0.930.47 ± 0.744.517b0.86 ± 0.920.47 ± 0.754.865b
Item 150.99 ± 1.030.58 ± 0.734.766b1.04 ± 1.010.56 ± 0.765.680b
Item 160.66 ± 0.880.19 ± 0.486.864b0.65 ± 0.880.23 ± 0.555.903b
Item 170.59 ± 0.830.26 ± 0.524.818b0.59 ± 0.820.28 ± 0.574.597b
Item 180.65 ± 0.890.19 ± 0.446.849b0.58 ± 0.860.29 ± 0.584.145b
Item 190.96 ± 1.040.46 ± 0.835.430b1.01 ± 1.150.45 ± 0.686.188b
Item 200.80 ± 1.050.40 ± 0.884.268b0.93 ± 1.160.31 ± 0.666.807b
Item 210.84 ± 1.000.43 ± 0.874.526b0.86 ± 1.100.45 ± 0.754.491b
Item 220.60 ± 0.860.41 ± 0.662.551c0.61 ± 0.860.41 ± 0.672.757b
Item 230.73 ± 1.040.23 ± 0.496.323b0.66 ± 1.030.32 ± 0.624.235b
Item 240.87 ± 1.080.37 ± 0.615.792b0.91 ± 1.100.37 ± 0.626.321b
Item 251.06 ± 1.070.72 ± 1.103.371b1.23 ± 1.180.59 ± 0.886.351b
Item 260.64 ± 0.960.21 ± 0.545.724b0.62 ± 0.960.26 ± 0.594.732b
CCL-D7.42 ± 6.873.67 ± 4.256.724b7.18 ± 6.854.14 ± 4.765.344b
CCL-A9.40 ± 9.384.44 ± 5.556.597b9.69 ± 9.284.51 ± 5.946.918b

aData are presented as mean ± SD.

bP < 0.05.

cP < 0.01.

dP = 0.07.

The strong correlation between the subscales of the CCL (r = 0.73) also shows that there is substantial overlap in variance. Therefore, we also calculated partial correlations between the CCL subscale and one of the mood severity rating scales. After controlling for the remaining scale, partial correlations between each CCL subscale and the same-affect rating scale remained significant (Ps < 0.05), whereas the correlations between the opposite-affect rating scales became non-significant. Further, the partial correlations with the mood severity were only statistically significantly different for the depression subscale (Z = 3.15, P < 0.01). The partial correlations are shown in Table 3.

5. Discussion

NATs are hypothesized to be central in the development and treatment of psychopathology according to the theory by Beck (1, 13, 27). Yet, there is a limited number of scales that can be utilized to assess NATs in Turkish (50). Moreover, with the available scales it is not possible to assess for both depressive and anxious negative cognitions simultaneously. Therefore, the current study aimed to evaluate the psychometric properties of the Turkish version of the CCL, and the results in a psychiatric outpatient sample revealed that the scale had excellent internal consistency and good test-retest reliability, that it consisted of two discrete factors, and that it provided evidence for concurrent and discriminant validity.

The two-factor structure of the CCL is in line with the previous reports of the psychometric properties of the CCL. To a great extent, factor loadings of the individual items corresponded with the expected subscale of the CCL. The two items that loaded more strongly on the opposite-affect factor were decidedly left on their original subscales, as they were more relevant to these subscales according to their themes, i.e., item 14 and negative self-evaluation, item 26 and physical threat. Since five items of the CCL cross-loaded on both subscales, after the investigation of the content of these items, it was decided that they should be classified into the originally proposed subscales. The four items that failed to load on their respective subscales fell only quite circumstantially shorter than the expected values. Yet, items 1, 8, 14, and 26 were the most frequently problematic items, as listed above. Therefore, further investigation of these items in subsequent studies is warranted.

The CCL subscales were found to correlate positively with the measures of mood severity and the ATQ. First, the subscales’ association with the ATQ demonstrated that the CCL was able to successfully assess the same construct. Second, although the CCL subscales failed to selectively correlate with the same-affect measures of severity, the significant difference in the partial correlation between the CCL-D and the HADS depression subscale, when compared with the correlation between the CCL-A and the HADS anxiety subscale, provided further evidence that the CCL had both concurrent and discriminant validity. The latter was also substantially validated by the group comparisons, where the participants were classified according to cut-off scores of depression and anxiety.

The stronger and statistically different correlation between the CCL-D and the ATQ compared to the correlation between the CCL-A and the ATQ might be interpreted as a finding for the highly prototypical cognitive content of the depression items, whereas as previously reported in the literature (30), the anxiety items of the CCL might be suggestive of more general, less prototypical cognitive content. This finding might have some clinical implications, such that patients with depression and comorbid anxiety disorders would score high on both subscales (26, 40, 41, 60). On the other hand, depressed patients with no comorbid anxiety would score lower on the anxiety items of the CCL. This might be indicative for a differential diagnosis of depressed patients with and without anxiety disorders, as suggested by Beck (10, 37). Yet, the high scores on the anxiety subscale would not permit the diagnosis of a specific anxiety disorder, and further evaluation with a highly prototypical cognitive content for anxiety would be necessary, directed by the clinical presentation of the patients.

The strong correlation between the subscales of the CCL indicates that there is substantial overlap in the items for assessing negative cognitive content related to depression and anxiety. Although the cognitive content specificity hypothesis posits that NATs may enhance the probability of correct differential diagnosis (10, 37), results of this study have only partially supported this claim, which is in accordance with the literature (15). However, the main focus of the current study was not to evaluate the cognitive content specificity hypothesis, which would require additional assessments.

The current study is the first to provide evidence that the CCL has sound psychometric properties in a Turkish outpatient sample. Further, the CCL is the second scale available to assess negative depressive cognitions in Turkish; with its more comprehensive structure, which also allows for the assessment of anxiety related NATs, it is likely that it might provoke more research ideas focusing on the effect of NATs in the differential diagnosis, development, maintenance, and treatment of mental disorders.

The limitations of the study may be summarized as follows: (i) the limited number of participants, (ii) the limited number of psychopathology scales, (iii) the participants’ low scores on the depression and anxiety rating scales, (iv) the lack of an anxiety-specific negative cognition scale for concurrent validity analysis, (v) the comorbid psychiatric diagnoses of the participants, and (vi) the use of only self-report measures.

In conclusion, the current study has provided evidence that the Turkish version of the CCL is a reliable and valid instrument to assess NATs in a clinical outpatient sample.

Acknowledgements

References

  • 1.

    Beck AT. Cognitive therapy and the emotional disorders. Penguin; 1979.

  • 2.

    Beck AT, Dozois DJ. Cognitive therapy: current status and future directions. Annu Rev Med. 2011;62:397-409. [PubMed ID: 20690827]. https://doi.org/10.1146/annurev-med-052209-100032.

  • 3.

    Beck AT, Haigh EA. Advances in cognitive theory and therapy: the generic cognitive model. Annu Rev Clin Psychol. 2014;10:1-24. [PubMed ID: 24387236]. https://doi.org/10.1146/annurev-clinpsy-032813-153734.

  • 4.

    Hofmann SG, Asmundson GJ, Beck AT. The science of cognitive therapy. Behav Ther. 2013;44(2):199-212. [PubMed ID: 23611069]. https://doi.org/10.1016/j.beth.2009.01.007.

  • 5.

    Butler AC, Chapman JE, Forman EM, Beck AT. The empirical status of cognitive-behavioral therapy: a review of meta-analyses. Clin Psychol Rev. 2006;26(1):17-31. [PubMed ID: 16199119]. https://doi.org/10.1016/j.cpr.2005.07.003.

  • 6.

    Cuijpers P, Karyotaki E, Weitz E, Andersson G, Hollon SD, van Straten A. The effects of psychotherapies for major depression in adults on remission, recovery and improvement: a meta-analysis. J Affect Disord. 2014;159:118-26. [PubMed ID: 24679399]. https://doi.org/10.1016/j.jad.2014.02.026.

  • 7.

    McMain S, Newman MG, Segal ZV, DeRubeis RJ. Cognitive behavioral therapy: current status and future research directions. Psychother Res. 2015;25(3):321-9. [PubMed ID: 25689506]. https://doi.org/10.1080/10503307.2014.1002440.

  • 8.

    Beck AT. The current state of cognitive therapy: a 40-year retrospective. Arch Gen Psychiatry. 2005;62(9):953-9. [PubMed ID: 16143727]. https://doi.org/10.1001/archpsyc.62.9.953.

  • 9.

    Clark DA, Beck AT. Cognitive theory and therapy of anxiety and depression. In: Kendall C, Watson D, editors. Anxiety and Depression: Distinctive and Overlapping Features. San Diego: Academic Press; 1989. p. 379-411.

  • 10.

    Beck AT, Brown G, Steer RA, Eidelson JI, Riskind JH. Differentiating anxiety and depression: a test of the cognitive content-specificity hypothesis. J Abnorm Psychol. 1987;96(3):179-83. [PubMed ID: 3680754].

  • 11.

    Greenberg MS, Beck AT. Depression versus anxiety: A test of the content-specificity hypothesis. J Abnormal Psychol. 1989;98(1):9-13. https://doi.org/10.1037/0021-843x.98.1.9.

  • 12.

    Lamberton A, Oei TP. A test of the cognitive content specificity hypothesis in depression and anxiety. J Behav Ther Exp Psychiatry. 2008;39(1):23-31. [PubMed ID: 17217911]. https://doi.org/10.1016/j.jbtep.2006.11.001.

  • 13.

    Beck JS. Cognitive behavior therapy: Basics and beyond. Guilford Press; 2011.

  • 14.

    Kendall PC, Ingram RE. Cognitive-behavioral perspectives: Theory and research on depression and anxiety. In: Kendall PC, Watson DE, editors. Anxiety and Depression: Distinctive and Overlapping Features. San Diego: Academic Press; 1989. p. 27-53.

  • 15.

    Beck R, Perkins TS. Cognitive content-specificity for anxiety and depression: A meta-analysis. Cogn Ther Res. 2001;25(6):651-63.

  • 16.

    Higgins ET. Self-discrepancy: a theory relating self and affect. Psychol Rev. 1987;94(3):319-40. [PubMed ID: 3615707].

  • 17.

    Ghahramanlou-Holloway M, Wenzel A, Lou K, Beck AT. Differentiating cognitive content between depressed and anxious outpatients. Cogn Behav Ther. 2007;36(3):170-8. [PubMed ID: 17852174]. https://doi.org/10.1080/16506070701374256.

  • 18.

    Hendriks SM, Licht CMM, Spijker J, Beekman ATF, Hardeveld F, de Graaf R, et al. Disorder-specific cognitive profiles in major depressive disorder and generalized anxiety disorder. BMC Psychiatr. 2014;14(1):96. https://doi.org/10.1186/1471-244x-14-96.

  • 19.

    Ayers CR, Riskind JH. Testing the psychometric properties of the Cognitions Checklist, a measure to differentiate anxiety and depression among older adults. Geriatr Nurs. 2014;35(5):370-3. [PubMed ID: 24910292]. https://doi.org/10.1016/j.gerinurse.2014.05.002.

  • 20.

    Brown HM, Waszczuk MA, Zavos HM, Trzaskowski M, Gregory AM, Eley TC. Cognitive content specificity in anxiety and depressive disorder symptoms: a twin study of cross-sectional associations with anxiety sensitivity dimensions across development. Psychol Med. 2014;44(16):3469-80. [PubMed ID: 25066519]. https://doi.org/10.1017/S0033291714000828.

  • 21.

    Miranda R, Fontes M, Marroquin B. Cognitive content-specificity in future expectancies: role of hopelessness and intolerance of uncertainty in depression and GAD symptoms. Behav Res Ther. 2008;46(10):1151-9. [PubMed ID: 18687420]. https://doi.org/10.1016/j.brat.2008.05.009.

  • 22.

    Alford BA, Lester JM, Patel RJ, Buchanan JP, Giunta LC. Hopelessness predicts future depressive symptoms: A prospective analysis of cognitive vulnerability and cognitive content specificity. J Clin Psychol. 1995;51(3):331-9. https://doi.org/10.1002/1097-4679(199505)51:3<331::aid-jclp2270510303>3.0.co;2-t.

  • 23.

    Schniering CA, Rapee RM. The relationship between automatic thoughts and negative emotions in children and adolescents: a test of the cognitive content-specificity hypothesis. J Abnorm Psychol. 2004;113(3):464-70. [PubMed ID: 15311991]. https://doi.org/10.1037/0021-843X.113.3.464.

  • 24.

    Jolly JB, Dyck MJ, Kramer TA, Wherry JN. Integration of positive and negative affectivity and cognitive content-specificity: improved discrimination of anxious and depressive symptoms. J Abnorm Psychol. 1994;103(3):544-52. [PubMed ID: 7930054].

  • 25.

    Laurent J, Stark KD. Testing the cognitive content-specificity hypothesis with anxious and depressed youngsters. J Abnorm Psychol. 1993;102(2):226-37. [PubMed ID: 8315135].

  • 26.

    Clark DA, Beck AT, Stewart B. Cognitive specificity and positive-negative affectivity: complementary or contradictory views on anxiety and depression? J Abnorm Psychol. 1990;99(2):148-55. [PubMed ID: 2348008].

  • 27.

    Beck AT. Cognitive therapy of depression. Guilford press; 1979.

  • 28.

    Clark DA, Beck AT. Cognitive therapy of anxiety disorders: Science and practice. Guilford Press; 2011.

  • 29.

    Tellegen A. Structures of mood and personality and their relevance to assessing anxiety with an emphasis on self-report. In: Tuma AH, Maser JD, editors. Anxiety and the Anxiety Disorders. Hillsdale, NJ: Erlbaum; 1985. p. 681-706.

  • 30.

    Woody SR, Taylor S, McLean PD, Koch WJ. Cognitive specificity in panic and depression: Implications for comorbidity. Cogn Ther Res. 1998;22(5):427-43. https://doi.org/10.1023/a:1018717514803.

  • 31.

    Watson D, Clark LA. Negative affectivity: the disposition to experience aversive emotional states. Psychol Bull. 1984;96(3):465-90. [PubMed ID: 6393179].

  • 32.

    Watson D, Tellegen A. Toward a consensual structure of mood. Psychol Bull. 1985;98(2):219-35. [PubMed ID: 3901060].

  • 33.

    Hollon SD, Kendall PC. Cognitive self-statements in depression: Development of an automatic thoughts questionnaire. Cogn Ther Res. 1980;4(4):383-95. https://doi.org/10.1007/bf01178214.

  • 34.

    Mock RE, Crandell RA, Mesfin GM. Induced latency in pseudorabies vaccinated pigs. Can J Comp Med. 1981;45(1):56-9. [PubMed ID: 6268266].

  • 35.

    Kendall PC, Hollon SD. Anxious self-talk: Development of the Anxious Self-Statements Questionnaire (ASSQ). Cogn Ther Res. 1989;13(1):81-93. https://doi.org/10.1007/bf01178491.

  • 36.

    Chambless DL, Caputo GC, Bright P, Gallagher R. Assessment of fear of fear in agoraphobics: the body sensations questionnaire and the agoraphobic cognitions questionnaire. J Consult Clin Psychol. 1984;52(6):1090-7. [PubMed ID: 6520279].

  • 37.

    Steer RA, Beck AT, Clark DA, Beck JS. Psychometric properties of the Cognition Checklist with psychiatric outpatients and university students. Psychol Assess. 1994;6(1):67-70. https://doi.org/10.1037/1040-3590.6.1.67.

  • 38.

    Osman A, Besett TM, Osman JR, Troutman JA, Grittmann L. Systematic evaluation of psychometric properties of the Cognition Checklist with college students. Psychol Rep. 1995;76(2):523-8. [PubMed ID: 7667464]. https://doi.org/10.2466/pr0.1995.76.2.523.

  • 39.

    Clark DA, Beck AT, Brown G. Cognitive mediation in general psychiatric outpatients: a test of the content-specificity hypothesis. J Pers Soc Psychol. 1989;56(6):958-64. [PubMed ID: 2746459].

  • 40.

    Clark DA, Beck AT, Beck JS. Symptom differences in major depression, dysthymia, panic disorder, and generalized anxiety disorder. Am J Psychiatry. 1994;151(2):205-9. [PubMed ID: 8296890].

  • 41.

    Somoza E, Steer RA, Beck AT, Clark DA. Differentiating major depression and panic disorders by self-report and clinical rating scales: Roc analysis and information theory. Behav Res Ther. 1994;32(7):771-82. https://doi.org/10.1016/0005-7967(94)90035-3.

  • 42.

    Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, et al. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry. 1998;59 Suppl 20:22-33. [PubMed ID: 9881538].

  • 43.

    American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. Washington; American Psychiatric Association. 2000.

  • 44.

    Lecrubier Y, Sheehan DV, Weiller E, Amorim P, Bonora I, Sheehan KH, et al. The Mini International Neuropsychiatric Interview (MINI). A short diagnostic structured interview: reliability and validity according to the CIDI. Eur Psychiatr. 1997;12(5):224-31.

  • 45.

    Sidani S, Guruge S, Miranda J, Ford-Gilboe M, Varcoe C. Cultural adaptation and translation of measures: An integrated method. Res Nurs Health. 2010:n/a. https://doi.org/10.1002/nur.20364.

  • 46.

    Cha ES, Kim KH, Erlen JA. Translation of scales in cross-cultural research: issues and techniques. J Adv Nurs. 2007;58(4):386-95. [PubMed ID: 17442038]. https://doi.org/10.1111/j.1365-2648.2007.04242.x.

  • 47.

    Zettle RD, Webster BK, Gird SR, Wagener AL, Burdsal CA. Factor Structure of the Automatic Thoughts Questionnaire in a Clinical Sample. Int J Cogn Ther. 2013;6(3):280-91. https://doi.org/10.1521/ijct.2013.6.3.280.

  • 48.

    Dobson KS, Breiter HJ. Cognitive assessment of depression: Reliability and validity of three measures. J Abnormal Psychol. 1983;92(1):107-9. https://doi.org/10.1037/0021-843x.92.1.107.

  • 49.

    Hollon SD, Kendall PC, Lumry A. Specificity of depressotypic cognitions in clinical depression. J Abnorm Psychol. 1986;95(1):52-9. [PubMed ID: 3700847].

  • 50.

    Sahin NH, Sahin N. Reliability and validity of the Turkish version of the Automatic Thoughts Questionnaire. J Clin Psychol. 1992;48(3):334-40. [PubMed ID: 1602023].

  • 51.

    Zigmond AS, Snaith RP. The Hospital Anxiety and Depression Scale. Acta Psychiatrica Scandinavica. 1983;67(6):361-70. https://doi.org/10.1111/j.1600-0447.1983.tb09716.x.

  • 52.

    Aydemir Ö, Guvenir T, Kuey L, Kultur S. Validity and reliability of Turkish version of Hospital Anxiety and Depression Scale. Turk Psikiyatri Derg. 1997;8(4):280-7.

  • 53.

    IBM Corp. 2012.IBM SPSS statistics for windows.

  • 54.

    Kaiser HF. The Application of Electronic Computers to Factor Analysis. Educ Psychol Measur. 1960;20(1):141-51. https://doi.org/10.1177/001316446002000116.

  • 55.

    Horn JL. A Rationale and Test for the Number of Factors in Factor Analysis. Psychometrika. 1965;30:179-85. [PubMed ID: 14306381].

  • 56.

    O’connor BP. SPSS and SAS programs for determining the number of components using parallel analysis and Velicer’s MAP test. Behav Res Methods Instr Comput. 2000;32(3):396-402. https://doi.org/10.3758/bf03200807.

  • 57.

    Patil VH, Singh SN, Mishra S, Todd Donavan D. Efficient theory development and factor retention criteria. J Business Res. 2008;61(2):162-70. https://doi.org/10.1016/j.jbusres.2007.05.008.

  • 58.

    Patil VH, Singh SN, Mishra S, Donovan DT. Parallel analysis engine to aid determining number of factors to retain. [Computer software]. 2015. Available from: http://smishra.faculty.ku.edu/parallelengine.htm.

  • 59.

    Meng X, Rosenthal R, Rubin DB. Comparing correlated correlation coefficients. Psychol Bull. 1992;111(1):172-5. https://doi.org/10.1037/0033-2909.111.1.172.

  • 60.

    Grunhaus L, Pande AC, Brown MB, Greden JF. Clinical characteristics of patients with concurrent major depressive disorder and panic disorder. Am J Psychiatry. 1994;151(4):541-6. [PubMed ID: 8147451].