Assessing Mastery Through Psychotherapy: Psychometric Properties of the Persian Version of the Mastery Scale

1. Background

In recent decades, considerable researches have confirmed the efficacy of psychodynamic therapies, particularly in the field of short-term psychodynamic psychotherapy (1). The interesting point is that, by reviewing the literature which have compared the results of different psychotherapies, it was revealed that the efficacy of these therapies are equal in many cases and no form of psychotherapy was superior to the others (2, 3). It is almost certain that lack of a significant difference between the results of two psychotherapies in a study, does not imply that they are equal.

Shielder holds the view that these studies typically measure some changes in patients as treatment outcome that does not coincide with what psychodynamic psychotherapy aims to accomplish. In most of these studies, symptom reduction is considered as a treatment outcome; However, the main purpose of psychodynamic psychotherapy is more than reduction of symptoms (4) and psychodynamically oriented researchers maintain that the efficacy studies of psychotherapy, should assess the changes in inner capacities and resources (5) and this requires different tools.

Therefore, various scales have been sought for evaluating different constructs, in order to observe this psychological transition. One of them that appears to encompass the change process through psychotherapy is Mastery.

Mastery is a concept from Freud’s drive theory and refers to an ego function that helps a person not feel overwhelmed by his/her psychiatric symptoms and problems (6). Grenyer defines interpersonal mastery as ‘‘the development of self-control and self-understanding in the context of interpersonal relationships’’(7).

Lower levels of mastery can be identified by a person’s distressed reaction to a conflictual interpersonal situation, and higher levels of mastery can be determined by one’s efficient control over emotional reactions (6).

The mastery scale developed by Grenyer (1996) is one of the instruments used to assess level of mastery of transference-related interpersonal conflicts and changes in mastery through psychotherapy sessions (8). This scale has notable properties. Unlike questionnaires that have pre-determined content and answers, the narrations used in this scale are directly extracted from verbal samples from patients.

The Mastery Scale was developed for evaluating narratives about relationship interactions, which the patient defines during psychotherapy sessions, because it is assumed, that these narratives reflect his/her mastery and control in conflictual relationship (8).

The Mastery Scale has shown excellent inter-rater reliability (for 4 raters between 0.75 - 0.89) and test-retest reliability (between 0.86 - 0.97) in a sample of patients with mixed diagnoses (8) and good test-retest reliability in anxiety and personality disorder patients (r: 0.83) (9) in America. The validity of this scale was studied. It was found that changes in mastery level over the course of therapy were significantly related to changes in observer ratings of psychological health/sickness (8). Moreover, this scale had adequate validity and reliability among Italian clinical samples (10) and a sample of psychiatric patients in Korea (11). However, no Persian-translated version of the mastery scale is available, and the psychometric properties of the Mastery Scale in the Iranian population have not been assessed.

2. Objectives

The broad application of the Mastery scale, absence of a similar scale in Persian, and increased number of psychologists interested in psychoanalytic research, stimulated this study. Therefore, the aim of this research was to examine the psychometric properties of the Persian version of the Mastery Scale.

3. Materials and Methods

3.1. Sample and Procedure

In this study, relationship episodes were used for scoring mastery. Relationship Episode refers to parts of a session in which the patient talks clearly about relationship with others. The relationship episodes were selected randomly from 18 supportive expressive therapy sessions of six depressed patients, at different phases of therapy. Data were collected in a private psychology clinic in Mashhad, Iran (during spring and summer 2015). All patients whose relationship narratives were used for this study, gave written informed consent. Relationship episodes were collected through some convergent questions like: “please tell me some incidents, events or interactions, each involving you and another person” (8).

Due to ethical considerations, raters only had access to relationship episodes (not full content of the sessions), therefore, only the complete relationship episodes were used to provide less ambiguity for raters. According to the mastery scale manual, the relationship episodes are judged for completeness on a 5-point scale. A complete relationship episode should have a structure consisting of an introduction, a middle section containing details of the incident, and a final section where the story ends. In addition, the wish of the narrator, the reaction of others to his wish, and his reaction to this interaction must be expressed or implied (8).

Episodes were written specifically and all scoreable clauses (n: 1270) were determined, such that all raters scored equal clauses at all episodes. Relationship episodes were scored by three independent judges. Two of them were clinical psychologists; one had a doctorate degree (PhD) in clinical psychology. For test re-test reliability, episodes were scored again by raters after 3 weeks. Before scoring, each rater scored four episodes, and then the scores were compared and discussed. By the end of this process, judges were in agreement for more 85% of the time for the scores of each clause.

According to Bonett (12), the sample size for inter-rater reliability was estimated using three raters, a power of 80% and a significance level of 5%, 36 episodes were needed.

The adequacy of the sample size for test re-test reliability was assessed using power analysis and sample size software (PASS, 2014). The model was tested using repeated measures and model testing was performed with the subsample of n = 17. The initial power analysis showed that 29 episodes with significance level of 5%, could produce 80% power.

3.2. Scale

The mastery scale is a comprehensive research tool for assessing a patient’s mastery over transference-related interpersonal conflicts during the treatment process. The aim of developing this tool was to provide researchers with an accurate and reliable scale which can be used in various research projects with a different psychotherapy approach (8). The scale consists of 23 categories of self-understanding and self-control which are related to six levels of interpersonal mastery.

For scoring, the rater classified the clauses into 23 categories of mastery scale and recorded the respective score. The mastery score of an individual is the mean score of all clauses of each relationship episode. Mastery scores of 5 and 6 show an understanding of the behavior patterns and also, control over emotions in inter-personal conflicts. While, lower values imply impairment in self-awareness and lack of control (e.g. control over emotions) (9). To choose a scoreable relationship episode and identify clauses, a mastery scale has certain rules which are described in its guideline (8). Table 1 shows a brief summary of the mastery scale.

In creating the Persian version of the mastery scale, permission was first obtained from Professor Grenyer. Second, forward translation was conducted. Third, back-translation was performed by a bilingual person. Finally, it was reviewed and reconfirmed by two psychologists and the final translation was fixed.

3.3. Statistical Analysis

Data normality was examined by the Shapiro-Wilk test. The inter-rater reliability between three raters was assessed by calculating interclass correlation coefficient ICC (3.3), and test-retest reliability for each rater was assessed by Pearson’s r. Finally, statistical analysis was conducted using SPSS software for windows (version 23).

4. Results

4.1. Test Retest Reliability

A Pearson product moment correlation coefficient was computed to assess the test-retest reliability of the mastery scale. It was assessed by selecting 29 relationship episodes at random and presenting them to raters again after three weeks. A high degree of reliability was found between the first time and the second time for all the raters. The results for raters were as follows: Rater 1’s test-retest reliability r = 0.95, rater 2 r = 0.91, Rater 3 r = 0.80, such that the results were highly significant at P < 0.0001.

4.2. Inter Rater Reliability

To assess the Inter-rater reliability of the mastery scale, three trained and independent raters, scored 36 relationship episodes, then ICC (3, 3) was calculated. Table 2 shows the results of intraclass correlation coefficient (ICC) details. A high degree of reliability was found between the raters. Correlation coefficients between the raters were as follows: rater 1, 2 = 0.89, rater 1, 3 = 0.80, rater 2, 3 = 0.85. The average measure ICC was 0.84 with a 95% confidence interval from 0.72 to 0.90, which is considered a good level of agreement (13). This value indicates that 84% of the variance is true score variance while the remaining 16% reflects error.

5. Discussion

In recent years, the usefulness of psychodynamic therapies in the mental health system has been controversial (14-16), especially because of the historical belief that empirical evidence supporting the effectiveness of these treatments is low (2). Therefore, numerous studies have been carried out in recent years on the effectiveness of psychodynamic therapies, and the findings support the effectiveness of this therapy on mild and complex mental health problems (1). Meta-analyses show the impact of psychodynamic therapies on the symptoms of many psychiatric disorders, for example, personality disorders (17), Depression (18, 19), Schizophrenia (20, 21); and even in long-term psychoanalytic therapy (2, 22, 23). But Kazdin (24) believes that no evidence-based explanation can be offered on how and why the intervention of a psychodynamic psychotherapist can result in changes in patients, and more research is needed on the mechanisms of change and variables involved in the effectiveness of the psychodynamic therapy.

On the other hand, psychodynamic therapies have been of interest in Iran and just a few studies have been performed on its effectiveness in Iranian subjects. However, the consequences and effects of a new treatment should be studied to ensure its effectiveness in a country. As explained in the introduction, it is clear that the measures used to assess outcome should be matched with the nature of psychodynamic therapies. To the best of the authors’ knowledge, although a growing number of studies have assessed the efficacy of psychoanalytic therapy in Iran, only a few specialized scales have been translated into Persian and their psychometric properties studied. Yet, no Persian-translated version of the mastery scale or another scale for rating the content of psychotherapy sessions is available in Persian; hence, the present study is the first attempt in this regard. On the other hand, the results revealed that the scale had excellent inter-rater reliability and good test-retest reliability.

The results of this study are generally consistent with previous findings for the English version of the mastery scale. The inter-rater coefficient correlation between 4 raters was reported to be from 0.75 to 0.89 (8) and between 2 raters 0.83 (7) and the test-retest reliability after 3 weeks with 4 raters was reported to be between 0.77 and 0.92 (8).

To use this tool, it is very important for raters to know the rating and the rules of properly preparing transcripts. Prioritizing the surface of speech and its consistency with scale instances, is as important for consideration as the story itself. Hence, it is crucial that raters read the relationship episode completely before rating.

The sample used in this study was collected through psychotherapy sessions and only complete relationship episodes were used; however, this scale can be useful for incomplete episodes or contents collected outside psychotherapy sessions (8). All the episodes were related to therapy sessions having patients diagnosed with major depression without psychotic symptoms and with higher education degrees and average economic and social status. There was no severe disorder in form and process of their thought and its content was uncomplicated. There will probably be less agreement among raters in rating, when it comes to patients with disorganized and symbolic speech or with thought disorder.

However, this scale paves the way for deep research on psychodynamic therapies. Emotional self-regulation and intellectual self-understanding in inter-personal relationships are important mental capacities and useful indices for evaluating the outcomes of psychodynamic therapies (8). The application of mastery scale, especially in case studies, provides accurate observing and following of changes in the ability of controlling and managing emotions and relationships for researchers, during various stages of psychotherapy.

5.1. Conclusion

Following multiple randomized clinical trials on psychodynamic therapies, currently there is no doubt about the empirical support of these treatments. However, the results of clinical studies are not generalizable; meaning that the effectiveness of a psychotherapy method on individuals in a society does not guarantee its usefulness in other communities.

In recent years, Iranian psychotherapists have been more interested in psychodynamic therapies (25). It seems that widespread use of this therapy, closer examination of the mechanisms of changes, and consequences of psychodynamic therapies in Iranian subjects are necessary. This is not possible without proper tools and scales. What highlights the significance of the findings of this study is the necessity of Iranian researchers to access scales which have been developed specifically for psychodynamic studies. The Mastery scale is a comprehensive tool which was developed according to this objective and based on the concept of mastery in psychodynamic theories.

According to the results, the Persian-translated mastery scale is a reliable tool that can be used in assessing the efficacy of psychodynamics therapies.

This study confirmed only the reliability of the mastery scale. The validity of this scale was studied and approved by Grenyer (8); however, the validity of the Persian version has to be approved as well.

5.2. Limitations and Suggestions

The major limitation of this study was the use of a very homogenized sample of relationship episodes from psychiatric patients suffering from major depression disorder. Also, more studies are needed to assess the agreement among raters with narratives of patients suffering from various psychiatric disorders and also the general population.

Although, there are increasing researches on the psychometric properties of the Persian version of psychology scales, there are just a few appropriate Persian translated scales for psychodynamic research. More considerations for these scales will facilitate future psychodynamic efficacy researches in Iran.