Psychometric Properties of the Persian Version of the Social Anxiety - Acceptance and Action Questionnaire

1. Background

Social anxiety disorder (SAD) is an intense fear of shame or embarrassment in social or performance settings, and is usually characterized by an avoidance of these situations. This fear is often related to specific impairment or distress in different areas of life, including occupational, social and family settings (1). Recent studies have shown the effectiveness of acceptance and commitment therapy (ACT) in patients with SAD (1-7). These results are promising yet we need valid and reliable instruments based on ACT to show the effectiveness of these interventions. Experiential avoidance (EA) is one of the fundamental constructs of ACT. It refers to a tendency in human beings to change the form, frequency and sensitivity of internal situations and events (8). Furthermore, EA has a role in generation and maintenance of psychological pathologies such as anxiety disorders (9, 10).

There are questionnaires such as the acceptance and action questionnaire (AAQ) designed to assess EA, acceptance, or psychological flexibility (11). There are also other questionnaires designed to assess EA or Acceptance, for more adaptation to disorders and special issues. Context-specific indices of acceptance have arisen from the belief that general measures of acceptance may possess less utility than those that pertain to certain circumstances. Theoretically, individuals may find it difficult to report avoiding decontextualized thoughts and feelings, as listed in these nonspecific measures, and may find it easier to respond that they avoid specific internal events that are associated with particular settings. For example, a socially anxious individual may be able to more accurately report the avoidance of thoughts that are socially distressing rather than negative thoughts in general (12). Despite the growing body of complaint-specific acceptance measures, none have assessed acceptance specific to social anxiety. One of these specific questionnaires is the social anxiety - acceptance and action questionnaire (SA-AAQ) developed by MacKenzie and Kocovski (12) to assess acceptance specific to social anxiety symptoms or the extent to which an individual is aware of thoughts and feelings about their social anxiety without attempting to change them. Acceptance is where an individual is unwilling to remain in contact with internal experiences (such as thoughts and feelings), to Acceptance, where an individual actively experiences his/her internal events. The 16-item version of acceptance and action questionnaire (AAQ) is an important scale based on which the SA-AAQ was designed. The questions of this scale were changed for useful assessment of social anxiety. The first and the last versions of this questionnaire have 56 and 19 Likert-type items ranging from ‘never true’ to always true. Higher scores signify higher acceptance of social anxiety-related thoughts and feelings. Using factor analysis, MacKenzie and Kocovski (12) found two factors of this questionnaire. The first factor (acceptance) consisted of 13 items, with an eigenvalue of 9.55, and accounted for 50.28% of the variance. This factor included questions such as I worry about not being able to control my social anxiety. The second factor (action) comprised of six items, with an eigenvalue of 1.82, and accounted for 9.56% of the variance. This factor included questions such as I get on with my life even when I feel socially anxious. The SA-AAQ had negative relationships with measures of social anxiety. It also had positive relationships with measures of mindfulness, and a negative relationship with a measure of thought suppression. The SA-AAQ had an internal consistency of 0.94, assessed using the Cronbach’s alpha. In another study, Cantarinhas (13) reported good psychometric properties of the SA-AAQ in 599 adolescents aged 14 to 18 years. Valid and reliable instruments are needed to understand the mechanisms through which ACT works to create useful changes. Due to the lack of a valid and reliable instrument for assessing social anxiety-related acceptance and action in Iran, and the necessity of assessing the validity and reliability of instruments in different cultures and languages, this study and its usage in clinical and non-clinical populations, in both clinical and research areas, is important and necessary.

2. Objectives

The purpose of the present study was to examine the psychometric properties of the Persian version of the social anxiety-acceptance and action questionnaire (SA-AAQ) in university students.

3. Materials and Methods

This was a cross-sectional study. The statistical population included all the students of Shahid Beheshti University of Medical Sciences during the 2014 to 2015 academic year. Overall, 324 students were selected using the cluster random sampling method. Aliabadi believed that participant ratio should be double the number of variables, and it is better for this ratio to be higher (14). Thus, in this study the selected ratio was three to one. The students were included in the study after having received an explanation of the research process, providing an informed consent, and considering the ethical issues. Thirty individuals were chosen from this group to participate in the test-retest stage, after four weeks. We contacted one of the developers to obtain permission for translating the scale, and assessed the validity and reliability of the scale in Iran. Independently from each other, one of the researchers (PhD student in clinical psychology), an MA English student, and a PhD English student translated the SA-AAQ, and then, a unified version of these translations was back-translated to English by a English PhD student. The translated script was compared with the main script, and the mistakes were identified and rectified with the cooperation of one of the developers of the scale (MacKenzie). We asked four University experts to examine the scale in terms of relevancy, clarity and comprehensiveness, and to propose their suggestions. During the next stage, we used the scales on a sample of 20 students, and asked them to read the questions carefully, identify the unclear questions, and write their suggestions regarding the unclear questions. The final version was examined, verified, and made ready for use with the cooperation of one of the experts. We used the acceptance and action questionnaire (AAQ-II), social interaction anxiety scale (SIAS), cognitive fusion questionnaire (cfq), believability of anxious feelings and thoughts questionnaire (BAFT), valued living questionnaire (vlq), and the world health organization quality of life (WHOQOL) questionnaire to assess concurrence and convergent validity. The results were analyzed using the SPSS 17 software; correlational and factor analysis methods (principal components method along with varimax rotation) were used for analyzing the data.

3.1. Instruments

3.1.1. Social Anxiety-Acceptance and Action Questionnaire

Cronbach’s alpha was reported as 0.94 by the developers of the SA-AAQ. This questionnaire also has a good validity (12).

3.1.2. Acceptance and Action Questionnaire (AAQ-II)

This questionnaire was developed by Bond et al. (11), and is comprised of 10 questions assessing acceptance, experiential avoidance and psychological inflexibility. Higher scores indicate greater mental flexibility. Bond et al. (11) found good reliability, validity and construct validity for this scale. Abasi et al. (15), provided evidences for the psychometric adequacy of this questionnaire in Iran. Exploratory factor analysis revealed two factors: experiential avoidance of emotion and control over life. In four groups, the internal consistency and split-half coefficients were reported to be good (15).

3.1.3. Social Interaction Anxiety Scale (SIAS)

This questionnaire was developed by Heimberg et al. (16). It has 20 Likert-type questions about a person’s reactions to situations related to the group and interpersonal social interactions. Higher scores indicate higher levels of anxiety in social interactions. The validity and reliability of this questionnaire were reported as 0.84 and 0.91, respectively (16). Analyzed using the test-retest and Cronbach’s alpha methods, the reliability of this questionnaire was calculated as 0.79 and 0.90, respectively. Its validity was assessed using the brief fear of negative evaluation scale (BFNE) and the social phobia inventory (SPIN), and was reported as 0.54 and 0.68, respectively. Cronbach’s alpha and test-rest reliability for the Iranian version of SIAS was 0.90 and 0.79, respectively. The correlation between SIAS and BFNE and SPIN was 0.54 and 0.68, respectively (17).

3.1.4. Cognitive Fusion Questionnaire (CFQ)

This questionnaire was developed by Gillanders et al. (18), and includes seven Likert-type questions ranging from never true to completely true. Higher scores indicate highest cognitive fusion. Gillanders et al. (18) found good early evidences for the factor structure, reliability, stability over time, validity, discriminant validity and sensitivity to therapy. The test-rest reliability with a four-week interval was 0.81. In the present study, Cronbach’s alpha of this scale was calculated as 0.86.

3.1.5. Believability of Anxious Feelings and Thoughts Questionnaire (BAFT)

This questionnaire was developed by Herzberg et al. (19) in order to assess believability or cognitive fusion in people with anxiety disorders. Herzberg et al. (2012) reported the validity and reliability of this questionnaire to be good in a non-clinical sample, and a sample consisting of people with high anxiety. The results of factor structure analysis revealed three factors in this questionnaire, which were bodily symptoms, emotion regulation and negative evaluation. Internal consistencies of this questionnaire were reported to be 0.90 and 0.91, for healthy and anxious people, respectively. Test-retest reliability was reported as 0.77 for anxious people (19). In the present study, Cronbach’s alpha of this scale was calculated as 0.82.

3.1.6. Valued Living Questionnaire (VLQ)

This questionnaire is a two-section instrument developed by Wilson et al. (20) to assess valued living. In the first section, in a Likert-type scale, participants arranged 10 life domains in order of priority. The second section of this questionnaire asks respondents to rate on a Likert-type scale how consistently they have lived according to this pattern of valued behavior in every domain of life, during the previous week (20). In two studies, Cronbach’s alpha and Test-retest reliability for this questionnaire was reported as good. Construct and concurrent validity of this questionnaire were reported to be good, with problematic domains of life and psychological strong points (20). In the present study, the Cronbach’s alpha of this questionnaire was calculated as 0.84.

3.1.7. World Health Organization Quality of Life (WHOQOL)

This questionnaire assesses these four areas: physical health, psychological health, social relationships and environment. The studies done by the WHO has shown the suitability of this questionnaire in 40 countries of the world (21). Moreover, the psychometric properties of the Iranian version of this questionnaire indicate that it can be used in Iran (22).

Ethical considerations: The students were included in the study after having received an explanation of the research process, providing an informed consent, and considering the ethical issues.

4. Results

The present study was conducted on university students studying at a university of medical sciences during academic years 2014 to 2015. The sample size was 324 students (135 males and 189 females). The distribution of the participants according to the field of study was 109 (33/6 percent) at the school of medical education sciences, 80 (24.7 percent) at the school of nursing and midwifery, 57 (17.6 percent) at the school of nutrition sciences and food technology, 56 (17.3 percent) at the school of health, and 22 (6.8 percent) at the school of pharmacy. The sample ranged in age from 18 to 43 with mean age of 22.01 (SD = 1.69). Two hundred and eighty six (88.3%) students were single and 38 (11.7%) were married (Table 1). All students were administered a questionnaire that included the AAQ-II, SIAS, CFQ, BAFT, VLQ and the WHOQOL. In order to obey the ethical principles of the research, the research questionnaires were submitted to the subjects of the study after assuring their volunteer attendance. In order to examine the validity of the SA-AAQ, we used different methods, such as content validity, construct validity, convergent validity and concurrent validity. Four psychology experts verified the content validity of the questionnaire.

4.1. Factor Analysis

In the analysis of the items, KMO and Bartlett’s sphericity tests indicated enough evidences for performing factor analysis. The KMO coefficient and chi-square of Bartlett’s sphericity test were calculated as 0.88 and 2.210 E3, respectively, and were significant at the 0.0001 level. We used a scree plot to determine the number of factors. The scree plot revealed three factors as components of the SA-AAQ. These factors accounted for 51.82% of the variance of the respective variable. The results of factor analysis and the factor loadings (for every item on every factor) are shown in Table 2. The minimal item loading on a factor was set at > 0.30. The first factor was named acceptance; items 5, 6, 7, 8, 12, 13, and 14 assess this factor. The second factor was named non-judgmental experience; this factor includes questions 15 - 19, and 10. The third factor was named action; this factor assesses questions 1 - 4, and 9 and 11. The results of factor analysis revealed that acceptance with an eigenvalue of 3.75, accounts for 19.73% of the total variance; non-judgmental experience with an eigenvalue of 3.73 accounts for 17.75% of the total variance; Action with an eigenvalue of 2.72, accounts for 14.32% of the total variance. These three factors together account for 51.82% of the total variance of the sample.

4.2. Convergent and Divergent Validity

Table 3 shows the correlations between the factors of the SA-AAQ, and between each factor and the total scale score. According to the results, the correlations between each factor of the SA-AAQ and the total scale score were much stronger than the correlations between the factors. These correlations indicate the discriminant and convergent validity of the SA-AAQ.

There was a negative relationship between the SA-AAQ and its subscales and the AAQ-II. This means that as the scores on the SA-AAQ or the acceptance of the social anxiety-related thoughts and feelings increase, experiential avoidance decreases. There is also a significant negative relationship between social anxiety-related acceptance and action and anxiety in social interactions. This means that as the acceptance of social anxiety symptoms increase, the scores on the SIAS decrease, and the person reports lower anxiety in social interactions. There is also a negative significant relationship between the SA-AAQ and two subscales with two measures of cognitive fusion (BAFT and CFQ). There was no significant relationship between the Action subscales of SA-AAQ with the cognitive fusion. This means that as the acceptance of social anxiety symptoms increases, the amount of cognitive fusion or the believability of the anxious feelings and thoughts decreases. Also, there was no significant relationship between the SA-AAQ and the VLQ, but there was a positive significant relationship between the Action subscales and the VLQ. There was a significant relationship between the SA-AAQ and its subscales and the WHOQOL. This means that as the acceptance of the social anxiety symptoms increases, the quality of life also increases (Table 4).

4.3. Reliability

Cronbach’s alpha coefficient for the total score, acceptance factor, nonjudgmental experience factor, and Action factor was 0.84, 0.87, 0.78, and 0.70, respectively. This finding indicates the high internal consistency of this questionnaire. The test-retest reliability (after four weeks) total score, acceptance factor, non-judgmental experience factor, and the action factor was 84, 0.81, 0.70 and 0.75, respectively.

5. Discussion

The aim of this study was to examine the validity and reliability of the SA-AAQ for the student of Shahid Beheshti University of Medical Sciences. This questionnaire has been designed to assess acceptance of the symptoms of social anxiety. The factor analysis showed that 19 questions of the SA-AAQ load on three factors. The first factor was named acceptance, the second factor was named nonjudgmental experience, and the third factor was named action. The findings of the acceptance and action factors were consistent with the findings of MacKenzie and Kocovski, (12), and Cantarinhas (13) yet, we found another factor, i.e. the ‘Nonjudgmental experience’ which is inconsistent with the findings of these studies. nonjudgmental experience included questions 19, 18, 16, 17, 10 and 15. Since questions 15 - 18, and 19 were taken from The five facet mindfulness questionnaire (FFMQ) (23), this finding is not surprising. This is not true for question 10, however this question was considered as a part of the nonjudgmental acceptance dimension. nonjudgmental experience refers to a state of not judging the internal experiences (23). It is worthwhile to say that when we look for two factors, the acceptance and action factors are acquired, and the questions of each factor are consistent with the questions designed by the developers of the scales. Therefore, questions 1, 2, 3, 4, 9, and 11 were loaded on the action factor, and the other questions were loaded on the acceptance factor, which accounts for 44.61% of the respective variance. The first factor with an eigenvalue of 5.70 accounted for 30.04% of the variance, and the second factor with an eigenvalue of 14.56, accounted for 2.76% of the variance. We can consider this as a two or three-factor questionnaire. If we consider it as a two-factor questionnaire, the questions of the nonjudgmental Experience factor load on the acceptance factor. As you can see, none of the questions, 19, 18, 16, 17, 10, and 15, load on the action factor, but all these questions load on the acceptance factor, or to be more accurate, on the nonjudgmental experience factor, which is one of the factors of mindfulness. This shows that we can consider the nonjudgmental experience factor as a part of the acceptance factor.

The internal consistency analysis provided evidences for the convergent and divergent validities of the SA-AAQ. According to these results, the correlations between the subscales were weaker than the correlations between the subscales and the total score. The convergent validity analysis of the SA-AAQ revealed significant negative relationships between this questionnaire and subscales with the AAQ-II, SIAS, and two measures of cognitive fusion; and a significant positive relationship between this questionnaire and the WHOQOL. No significant relationship was found between the SA-AAQ and subscales with the VLQ (with exception of the Action subscale). These findings are consistent with the findings of MacKenzie and Kocovski (12), who found a significant relationship between the SA-AAQ and the measures of social anxiety. However, in the present study, we only used the SIAS. We also found a negative significant relationship between the SA-AAQ and the SIAS; a finding which is different from the findings of MacKenzie and Kocovski (12). This difference is due to different scoring systems, meaning that the different scoring systems in this study caused the correlations mentioned as negative rather than positive. Another finding of this study, which was inconsistent with the findings of the developers of the scale was that the relationships between the SA-AAQ and the measures of social anxiety were stronger than the relationships between the SA-AAQ and the AAQ-II. This finding could be related to the measures, our sample or an overlap between the constructs of ACT. We only used one of the measures of social anxiety, and our sample was different. The strong negative relationship between the SA-AAQ and the two cognitive fusion questionnaires confirm an overlap between the constructs of ACT. As Gillanders et al. (18) pointed out, there is an overlap between the questions of these two questionnaires, and because the AAQ-II assesses psychological inflexibility, and because cognitive fusion is one of the components of cognitive inflexibility, the positive relationship between the two questionnaires seems reasonable. This is also consistent with the model of acceptance and commitment therapy, which considers cognitive fusion as the root of experiential avoidance. The negative relationship between the CFQ and the SA-AAQ can also be explained in this way. In this study, we found a positive significant relationship between the SA-AAQ and the WHOQOL, but there was no significant relationship between the subscales of SA-AAQ and the VLQ. This finding indicates that as the acceptance of social anxiety symptoms increases, the quality of life also increases.

The results also revealed a good reliability for the SA-AAQ. The total Cronbach’s alpha coefficient was calculated as 0.84. The original study and a study by Cantaharis (13) reported the internal consistency of the SA-AAQ to be 0.94 and 0.90, respectively. This finding indicates the high reliability of this questionnaire. The test-retest reliability analysis also showed the high reliability of this questionnaire.

5.1. Conclusion

The Farsi version of SA-AAQ had good validity and reliability when used on the Shahid Beheshti University of Medical Sciences students, and it can be used in research and therapy as a valid and reliable instrument. The present study was conducted on a sample that comprised of the students of a University, so we should be cautious about generalizing the results to other populations. Future studies can examine the validity and reliability of this questionnaire in samples comprised of people with social anxiety disorder.