1. Background
Social cognition is defined as the social performances based on social foundations such as understanding, interpretation, and response to the goals, interests, and behaviors of others (1). In other words, social cognition is the ability to recognize self and others and understand their mental state based on their tone of voice and facial/bodily appearances, as well as the ability to interpret their mental state by using the rules and knowledge of social affairs to interact with others and manage one’s emotions in interpersonal relationships (2-4).
Social cognition encompasses a wide range of behaviors related to understanding the emotions and mental state of others (4, 5). Disordered cognitive abilities could cause problems in interpersonal relations and interactions with the environment (6). Social cognition is a multifaceted construct consisting of different and dissociable, yet interrelated processes (7). Various domains of social cognition include the theory of mind (8), emotional processing, social understanding, and attritional styles (9, 10). Moreover, social cognition has been defined as the ability to evaluate the intentions, interests, and beliefs of others. Deficient social cognition is observable in various domains.
The precise and efficient identification of deficient social cognition is the first step toward effective medical planning. As such, the psychometric evaluation of social cognition is of paramount importance. Self-report scales are an alternative method for the identification of deficient social cognition. The observable social cognition rating scale (OSCARS) is an eight-item scale, which could be completed within five minutes, and the higher score indicate the higher deficiency in social cognition. In this regard, the study by Healey et al. (11) was conducted in the form of an early evaluation of this scale on 62 patients with schizophrenia and 50 healthy individuals, and the obtained results showed the high reliability and validity of this scale.
In another study, Halverson et al. (12) evaluated the psychometric characteristics of the OSCARS in patients with schizophrenia. In addition to confirming the psychometric characteristics of this scale, the results of the mentioned study demonstrated that the OSCARS is a beneficial tool for physicians for the early diagnosis of performance deficiencies and the need for psychological interventions. Given the importance of the social cognition construct in psychology, it is essential to examine the psychometric characteristics of this scale in Iran to find treatments and optimize lifestyles.
2. Objectives
The present study aimed to assess the validity and reliability of the Persian version of the OSCARS in Iran.
3. Methods
This was a descriptive-correlational study.
3.1. Participants
The sample population included the male and female students of the high schools in Dezful, Iran during 2019 - 2020. By using Cochran’s formula, 250 students aged 14 - 18 years (mean age: 15.88 ± 0.93 years) were selected via multistage random sampling. The current research was conducted after obtaining the required permit from the original developer of the OSCARS for translation into Persian, consultation with several psychology professors, and performing a pretest on some students to determine its readability. The students were provided with the required information about the objectives of the study and their participation before completing the questionnaire. At the next stage, the students were asked to state their agreement on a seven-point Likert scale. Each item of the questionnaire was scored within the range of 1 - 7.
After obtaining approval from the ethics committee of Shahid Chamran University of Ahvaz, data collection work began. All the participants completed the OSCARS and cognitive abilities questionnaire, which was used to examine congruent validity. Moreover, teachers completed the teacher’s report form for 56 students simultaneously and randomly. In order to measure validity, confirmatory factor analysis and congruent validity were used. The reliability of the tool was also measured using the Cronbach’s alpha and split-half method (Spearman-Brown and Guttman).
3.2. Research Instruments
3.2.1. OSCARS (Self-Report Form and Teacher’s Report Form)
The OSCARS has eight items to evaluate social cognition, and each item has one question, which targets one of the domains of social cognition. After each question, a behavioral example is presented to show a deficiency in that domain. Subsets of social cognition include the theory of mind, emotional perception, cognitive rigidity, jumping to conclusions, and attritional style. The participants would score their cognitive ability on a seven-point scale, and higher scores showed deficient social cognition.
The validity of the scale was set within the range of 0.50 - 0.70 using the retest method. In the study by Silberstein et al. (13), the internal consistency of the OSCARS was reported to be 0.80. The validity and reliability of this tool were first measured in the present study in Iran. Accordingly, the reliability of the OSCARS was estimated at 0.60 for the self-report version and 0.65 for the teacher’s version.
3.2.2. Cognitive Abilities Questionnaire
This questionnaire has been developed by Nejati to measure cognitive abilities (14). The tool consists of 30 items and seven micro-components of active memory, self-control and selective attention, decision-making, planning, stable attention, social cognition, and flexibility, which indicate that daily life situations require cognitive abilities. The items in the cognitive abilities questionnaire are scored based on a five-point Likert scale (never = 1, always = 5). In the study by Nejati (14), the reliability of this questionnaire was reported to be 0.83 based on the Cronbach’s alpha. This tool was used to examine the congruent validity of the OSCARS in the current research, and its validity was estimated at 0.85 based on the Cronbach’s alpha.
3.3. Statistical Analysis
Data analysis was performed in SPSS version 22 and AMOS version 22. Preliminary analyses (mean, standard deviation, and Pearson’s correlation-coefficient) were performed to gain an initial insight into the data. In addition, confirmatory factor analysis was performed in the AMOS software, and model retention indices were reported as well.
4. Results
Mean and Standard Deviation (SD) of scores of students and teachers in the Observable Social Cognition Rating Scale is presented in Table 1.
Table 1. Demographics Mean and Standard Deviation of Participants
| Variable | Range | Full Sample (N = 320), Mean ± SD | Female (N = 140), Mean ± SD | Man (N = 110), Mean ± SD |
|---|---|---|---|---|
| Age | 14 - 18 | 15.88 ± 0.93 | 15.82 ± 0.95 | 16.03 ± 0.95 |
| CGPA | 13.50 -20 | 18.73 ± 1.37 | 18.60 ± 1.33 | 18.05 ± 1.70 |
| OSCARS | 14 - 56 | 26.04 ± 3.80 | 27.81 ± 7.17 | 29.77 ± 5.95 |
| OSCARS (teacher’s version) | 23 - 46 | 32.35 ± 2.05 | 32.47 ± 2.19 | 32.17 ± 1.122 |
| CAQ | 33 - 90 | 68.45 ± 5.61 | 68.68 ± 5.74 | 67.72 ± 4.82 |
| Memory | 6 - 16 | 10.41 ± 1.06 | 10.44 ± 1.12 | 10.31 ± 0.52 |
| Inhibitory control | 6 - 33 | 14.17 ± 1.79 | 14.24 ± 1.89 | 13.95 ± 1.34 |
| Decision | 5 - 17 | 10.44 ± 1.15 | 10.42 ± 1.19 | 10.39 ± 0.79 |
| Planning | 3 - 12 | 6.26 ± 1.12 | 6.32 ± 1.21 | 6.16 ± 0.64 |
| Sustain attention | 3 - 14 | 7.71 ± 1.20 | 7.74 ± 1.26 | 7.57 ± 0.69 |
| Social cognition | 3 - 15 | 10.23 ± 1.22 | 10.26 ± 1.22 | 10.22 ± 0.89 |
| Cognitive flexibility | 4 - 17 | 9.19 ± 1.34 | 9.24 ± 1.42 | 9.08 ± 0.91 |
Abbreviations: CAQ, Cognitive Abilities questionnaire; CGPA, cumulative grade-point average; OSCARS, observable social cognition rating scale; y, years.
Table 1 shows the mean scores of the students and teachers in the OSCARS. Accordingly, the mean total score of the students and teachers in the OSCARS was 26.04 ± 3.80 and 32.35 ± 2.05, respectively. In addition, the mean score of their cognitive abilities was 68.45 and 5.61, respectively. Confirmatory factor analysis was used to examine the validity of the OSCARS, and the obtained data from the questionnaire were analyzed using the AMOS software version 22. Table 2 shows the factorial load of the items of the OSCARS divided into two forms (self-report and teacher’s report).
Table 2.Factor Loading of Items of OSCARS by Using Confirmatory Factor Analysis
| Questions | Self-Report | Teacher Edition | |
|---|---|---|---|
| 1 | Recognizing other people’s emotions, particularly negative emotions (sadness, fear, and anger) based on facial expression, body language, and/or vocal tone and rate? | 0.60 | 0.31 |
| 2 | Interpreting social interactions in a malevolent, hostile manner? | 0.40 | 0.30 |
| 3 | Making decisions quickly (i.e., jumps to conclusions) without examining other evidence? | 0.32 | 0.31 |
| 4 | Being flexible in interpreting social situations? | 0.37 | 0.40 |
| 5 | Can change or correct their interpretation of social interactions when wrong? | 0.30 | 0.35 |
| 6 | Understanding subtle jokes, sarcasm, and insults in conversation? | 0.65 | 0.64 |
| 7 | Seeing things from the perspective of others (i.e., putting themselves in other people’s shoes)? | 0.60 | 0.58 |
| 8 | Understanding subtle social cues, hints, and indirect requests (an example of an indirect request is if your son/daughter wants a toy, but rather than say so directly, comments on how pretty it is. | 0.37 | 0.70 |
According to the information in Table 2, the highest factorial load belonged to items seven and one (0.60), and the lowest factorial load belonged to item five (0.30). Moreover, the highest factorial load belonged to item eight (0.70), and the lowest factorial load belonged to item two (0.30) in the teacher’s report form. All the items were maintained given the desirability of the factorial loads, the significance of their route coefficient (β), and their upward trend (0.3).
Table 3 shows the goodness-of-fit index of the pattern resulting from the confirmatory factor analysis, including Chi-square, degree of freedom, significance level, normed chi-square measure, root mean square error of approximation (RMSEA), goodness of fit index (GFI), adjusted goodness of fit index, normed fit index, comparative fit index (CFI), and incremental fit index (IFI). The results of the confirmatory factor analysis revealed the excellent fit of the model.
Table 3.Hypothesized, Modified and Final SEM Model Fit Based on Fit Indicators
| Fit Indicators | χ2 | df | χ2/df | GFI | AGFI | IFI | TLI | CFI | NFI | RMSEA |
|---|---|---|---|---|---|---|---|---|---|---|
| Final model (self report) | 148.074 | 28 | 5.28 | 0.96 | 0.92 | 0.97 | 0.99 | 0.98 | 0.88 | 0.02 |
| Final model (teacher) | 276.024 | 17 | 16.23 | 0.90 | 0.80 | 0.80 | 0.81 | 0.80 | 0.71 | 0.1 |
In the current research, the RMSEA value was estimated at 0.02, which indicated that the data had an appropriate fit. The other indices of model fit also showed the excellent fit of the model (GFI = 0.96, CFI = 0.99, IFI = 0.99). Moreover, the results of confirmatory factor analysis regarding the teacher’s reports showed that all the factorial loads were higher than 0.3. However, indices of fit did not show a good fit, which could be due to the small sample size (n = 56). Pullman (15) states that RMSEA is not a good index for evaluating the fit of the model in the cases where the sample size is smaller than 150 since in this form, the value of this index is very high, and the value of other indices should be considered as well. Therefore, it is recommended that further investigation be conducted on appropriate sample sizes.
In the current research, in addition to conducting confirmatory factor analysis for the OSCARS, the questionnaire was used simultaneously with the cognitive abilities questionnaire to examine congruent validity. As mentioned earlier, 56 participants were selected randomly and completed the two questionnaires. Table 4 shows the correlation of the student’ scores in the OSCARS and cognitive abilities questionnaire.
Table 4.The Correlation Coefficient for Scores of Students for the OSCARS and Cognitive Abilities Questionnairea
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | |
|---|---|---|---|---|---|---|---|---|---|---|
| OSCARS | 1 | |||||||||
| OSCARS (teacher’s version) | 0.32** | 1 | ||||||||
| CAQ | 0.15* | 0.40** | 1 | |||||||
| Memory | 0.17** | 0.34** | 0.62** | 1 | ||||||
| Inhibitory control | 0.08 | 0.30** | 0.70** | 0.38** | 1 | |||||
| Decision | 0.11 | 0.21** | 0.61** | 0.18** | 0.34** | 1 | ||||
| Planning | 0.07 | 0.19** | 0.64** | 0.43** | 0.33** | 0.27** | 1 | |||
| Sustain attention | 0.03 | 0.15* | 0.72** | 0.35** | 0.32** | 0.50** | 0.30** | 1 | ||
| Social cognition | 0.13* | 0.40** | 0.27** | 0.01 | 0.04 | -0.03 | -0.006 | 0.20** | 1 | |
| Cognitive flexibility | 0.03 | 0.22** | 0.77** | 0.44** | 0.38** | 0.43** | 0.58** | 0.55** | 0.02 | 1 |
aN = 250; *, P < 0.05; and **, P < 0.01.
According to the information in Table 4, the correlation between the OSCARS scores of the students and teacher’s report form was 0.35. Moreover, the correlation with the cognitive abilities questionnaire was considered significant at 0.15, as well as with the social cognitive subscale (0.13; P < 0.001). The correlation between the subscales of the cognitive abilities questionnaire and the OSCARS has also been presented in the table above.
Two questions examined the validity of the two forms (self-report and teacher’s report) in Table 5.
Table 5.Correlation Results of Single Questions with the Whole Test
| Single Question | |
|---|---|
| Self-report | 0.30**; P < 0.001 |
| Teacher report | 0.25**; P < 0.001 |
Table 4 also shows the correlation of the questions with the two forms of the questionnaire (self-report and teacher’s report). In order to assess the validity of the OSCARS, we used the Cronbach’s alpha split-half method (Spearman-Brown and Guttmann), and the data of the entire samples (n = 250) were analyzed (Table 6).
Table 6.Reliability Coefficients OSCARS and Its Subscales by Cronbach’s Alpha Method and Splitting Method
| Variable | Reliability | ||
|---|---|---|---|
| Cronbach’s Alpha | Split-Half | ||
| Spearman-Brown | Guttman | ||
| OSCARS | 0.65 | 0.60 | 0.65 |
| OSCARS (teacher’s version) | 0.60 | 0.60 | 0.70 |
| CAQ | 0.85 | 0.76 | 0.95 |
According to the information in Table 6, the reliability coefficient of the OSCARS was estimated at 0.65, 0.60, and 0.65 based on the Cronbach’s alpha, Spearman-Brown split-half method, and Guttmann split-half method, respectively. These coefficients indicated the relatively good reliability. The alpha coefficient value was also obtained for the teacher’s report form, with the cognition abilities estimated at 0.60 and 0.85, respectively.
5. Discussion
The present study aimed to examine the validity and reliability of the OSCARS in a sample of students. According to the obtained results, the OSCARS is an appropriate tool for the evaluation of social cognition in adolescents. Furthermore, the results of confirmatory factor analysis regarding the validity of the questionnaire revealed that all the items were maintained given the desirability of the factorial loads. The GFI of the model was also considered desirable.
The significant correlations between the OSCARS, teacher’s report form, and the cognitive abilities questionnaire confirmed the concurrent validity of this scale. We also applied the Cronbach’s alpha, Spearman-Brown split-half method, and Guttmann split-half method to evaluate the reliability of the OSCARS. Our findings in this regard are consistent with the studies by Healey et al. (11), Silberstein et al. (13), and Halverson et al. (12).
Several international studies have also examined the validity of the OSCARS, proposing consistent results with our findings. For instance, Healey et al. (11) measured the congruent validity of the OSCARS, reporting that this tool has high internal consistency and retest validity. Moreover, their findings indicated that the OSCARS could be used as a diagnostic tool for patients with social cognition deficiency. Halverson et al. (12) also proposed similar results and confirmed the congruent validity, external validity, and predictive validity of the OSCARS. Furthermore, this tool has been shown to be an effective diagnostic tool at the early stages of treatment.
In general, the results of the present study indicated that the Persian version of the OSCARS has desirable reliability and validity and could be an efficient tool for screening and early diagnosis in identifying individuals with cognitive problems at early stages and taking actions for treatment since it is believed that these characteristics are more responsive to treatment at the early stages of life.
Our study had some limitations. First, the sample population was limited to the age range of 14-18 years, and the findings cannot be generalized to younger children or children in other communities. Therefore, it is recommended that further investigations in this regard be performed on larger sample sizes, and interviews be conducted while completing the questionnaire. Second, we used a questionnaire to determine the background, and some of the participants refused to answer correctly or gave unreal answers to the questions in this respect. It is suggested that interviews and questionnaires be used simultaneously in further research. Consultants and physicians could use the OSCARS as a practical indicator for the identification of high-risk individuals as well.
5.1. Conclusions
According to the results, the OSCARS could be used as a reliable questionnaire for the measurement of social cognition in both the self-report and teacher’s report versions and in clinical and non-clinical communities.
