Assessing Mastery Through Psychotherapy: Psychometric Properties of the Persian Version of the Mastery Scale

authors:

avatar Zohreh Edalati Shateri 1 , avatar Fahimeh Fathali Lavasani ORCID 1 , * , avatar Ali-Asghar Asgharnejad Farid 1 , avatar Mahmoud Dehghani ORCID 1 , avatar Sahar Attar 2 , avatar Narges Barzegar 2

Department of Clinical Psychology, Center of Excellence in Psychiatry and Clinical Psychology- School of Behavioral Sciences and Mental Health (Institute of Tehran Psychiatry), Iran University of Medical Sciences, Tehran, Iran
Department of Psychology, Hikmat-e- Razavi Nonprofit and Private Institute of Higher Education, Mashad, Iran

how to cite: Edalati Shateri Z, Fathali Lavasani F, Asgharnejad Farid A, Dehghani M, Attar S, et al. Assessing Mastery Through Psychotherapy: Psychometric Properties of the Persian Version of the Mastery Scale. Iran J Psychiatry Behav Sci. 2018;12(1):e7930. https://doi.org/10.5812/ijpbs.7930.

Abstract

Background:

Considering the importance of assessing the range of phenomena that can change in psychodynamic therapy, and lack of appropriate assessment scale in Persian language, the present study assessed the reliability of the Persian mastery scale. The mastery scale is a comprehensive research tool for assessing a patient’s mastery of transference-related interpersonal conflicts during the treatment process.

Objectives:

The aim of this research was to examine the psychometric properties of the Persian version of the Mastery Scale.

Methods:

After preparing the Persian version of the mastery scale through the forward-backward translation process, a sample of 36 relationship episodes was rated by three independent judges. Data were collected in Mashhad, Iran in 2015. The psychometric properties of the instrument, including inter-rater reliability and test-retest analysis were assessed.

Results:

The test-retest correlations of the 3 judges were (0.80, 0.91 and 0.95), highly significant at P < 0.0001 and the inter-rater reliability indicated a good level of agreement (The ICC (3, 3) 0.84, 95% CI [0.72, 0.90]).

Conclusions:

The current study has provided evidence that the Persian mastery scale is a reliable instrument for assessing changes in inner capacities and resources through psychodynamic treatments. However, further studies are required to investigate the validity of this scale.

1. Background

In recent decades, considerable researches have confirmed the efficacy of psychodynamic therapies, particularly in the field of short-term psychodynamic psychotherapy (1). The interesting point is that, by reviewing the literature which have compared the results of different psychotherapies, it was revealed that the efficacy of these therapies are equal in many cases and no form of psychotherapy was superior to the others (2, 3). It is almost certain that lack of a significant difference between the results of two psychotherapies in a study, does not imply that they are equal.

Shielder holds the view that these studies typically measure some changes in patients as treatment outcome that does not coincide with what psychodynamic psychotherapy aims to accomplish. In most of these studies, symptom reduction is considered as a treatment outcome; However, the main purpose of psychodynamic psychotherapy is more than reduction of symptoms (4) and psychodynamically oriented researchers maintain that the efficacy studies of psychotherapy, should assess the changes in inner capacities and resources (5) and this requires different tools.

Therefore, various scales have been sought for evaluating different constructs, in order to observe this psychological transition. One of them that appears to encompass the change process through psychotherapy is Mastery.

Mastery is a concept from Freud’s drive theory and refers to an ego function that helps a person not feel overwhelmed by his/her psychiatric symptoms and problems (6). Grenyer defines interpersonal mastery as ‘‘the development of self-control and self-understanding in the context of interpersonal relationships’’(7).

Lower levels of mastery can be identified by a person’s distressed reaction to a conflictual interpersonal situation, and higher levels of mastery can be determined by one’s efficient control over emotional reactions (6).

The mastery scale developed by Grenyer (1996) is one of the instruments used to assess level of mastery of transference-related interpersonal conflicts and changes in mastery through psychotherapy sessions (8). This scale has notable properties. Unlike questionnaires that have pre-determined content and answers, the narrations used in this scale are directly extracted from verbal samples from patients.

The Mastery Scale was developed for evaluating narratives about relationship interactions, which the patient defines during psychotherapy sessions, because it is assumed, that these narratives reflect his/her mastery and control in conflictual relationship (8).

The Mastery Scale has shown excellent inter-rater reliability (for 4 raters between 0.75 - 0.89) and test-retest reliability (between 0.86 - 0.97) in a sample of patients with mixed diagnoses (8) and good test-retest reliability in anxiety and personality disorder patients (r: 0.83) (9) in America. The validity of this scale was studied. It was found that changes in mastery level over the course of therapy were significantly related to changes in observer ratings of psychological health/sickness (8). Moreover, this scale had adequate validity and reliability among Italian clinical samples (10) and a sample of psychiatric patients in Korea (11). However, no Persian-translated version of the mastery scale is available, and the psychometric properties of the Mastery Scale in the Iranian population have not been assessed.

2. Objectives

The broad application of the Mastery scale, absence of a similar scale in Persian, and increased number of psychologists interested in psychoanalytic research, stimulated this study. Therefore, the aim of this research was to examine the psychometric properties of the Persian version of the Mastery Scale.

3. Materials and Methods

3.1. Sample and Procedure

In this study, relationship episodes were used for scoring mastery. Relationship Episode refers to parts of a session in which the patient talks clearly about relationship with others. The relationship episodes were selected randomly from 18 supportive expressive therapy sessions of six depressed patients, at different phases of therapy. Data were collected in a private psychology clinic in Mashhad, Iran (during spring and summer 2015). All patients whose relationship narratives were used for this study, gave written informed consent. Relationship episodes were collected through some convergent questions like: “please tell me some incidents, events or interactions, each involving you and another person” (8).

Due to ethical considerations, raters only had access to relationship episodes (not full content of the sessions), therefore, only the complete relationship episodes were used to provide less ambiguity for raters. According to the mastery scale manual, the relationship episodes are judged for completeness on a 5-point scale. A complete relationship episode should have a structure consisting of an introduction, a middle section containing details of the incident, and a final section where the story ends. In addition, the wish of the narrator, the reaction of others to his wish, and his reaction to this interaction must be expressed or implied (8).

Episodes were written specifically and all scoreable clauses (n: 1270) were determined, such that all raters scored equal clauses at all episodes. Relationship episodes were scored by three independent judges. Two of them were clinical psychologists; one had a doctorate degree (PhD) in clinical psychology. For test re-test reliability, episodes were scored again by raters after 3 weeks. Before scoring, each rater scored four episodes, and then the scores were compared and discussed. By the end of this process, judges were in agreement for more 85% of the time for the scores of each clause.

According to Bonett (12), the sample size for inter-rater reliability was estimated using three raters, a power of 80% and a significance level of 5%, 36 episodes were needed.

The adequacy of the sample size for test re-test reliability was assessed using power analysis and sample size software (PASS, 2014). The model was tested using repeated measures and model testing was performed with the subsample of n = 17. The initial power analysis showed that 29 episodes with significance level of 5%, could produce 80% power.

3.2. Scale

The mastery scale is a comprehensive research tool for assessing a patient’s mastery over transference-related interpersonal conflicts during the treatment process. The aim of developing this tool was to provide researchers with an accurate and reliable scale which can be used in various research projects with a different psychotherapy approach (8). The scale consists of 23 categories of self-understanding and self-control which are related to six levels of interpersonal mastery.

For scoring, the rater classified the clauses into 23 categories of mastery scale and recorded the respective score. The mastery score of an individual is the mean score of all clauses of each relationship episode. Mastery scores of 5 and 6 show an understanding of the behavior patterns and also, control over emotions in inter-personal conflicts. While, lower values imply impairment in self-awareness and lack of control (e.g. control over emotions) (9). To choose a scoreable relationship episode and identify clauses, a mastery scale has certain rules which are described in its guideline (8). Table 1 shows a brief summary of the mastery scale.

Table 1.

Mastery Scale

ScoreComponentSample
Level 1. Lack of impulse control.
1AExpressions of being emotionally overwhelmed.I exploded.
1BReferences to immediacy of impulses.I have lost all control.
1CReferences to blocking defenses.It does not bother me.
1DReferences to ego-boundary disorders.I am subservient to you.
Level 2. Introjections and projection of negative affects.
2EExpressions of suffering from internal negative states.I am very upset about it.
2FExpressions indicative of negative projection on to others.It’s all his fault.
2GExpressions indicative of negative projection from others.They are not interested in me.
2HReferences to interpersonal withdrawal.I don’t initiate friendships.
2IExpressions of helplessness.I can’t help it.
Level 3. Difficulties in understanding and control.
3JExpressions of cognitive confusion.I don’t know.
3KExpressions of cognitive ambivalence.I suppose that it was like this.
3LReferences to positive struggle with difficulties.I want to get over this.
Level 4. Interpersonal awareness.
4MReferences to questioning the reactions of others.I asked him why he thought of this
4NReferences to considering the other’s point of view.I can see she was jealous.
4OReferences to questioning the reaction of the self.It was my fault really.
4PExpressions of interpersonal self-assertion.I stood up against the bully.
Level 5. Self-understanding.
5QExpressions of insight into repeating personality patterns of selfI am always worrying about this
5RMaking dynamic links between past and present relationshipsI saw him like he was my father.
5SReferences to interpersonal union.I felt respected by him.
5TExpressions of insight into interpersonal relations.He is as stubborn as I am.
Level 6. Self-control.
6UExpressions of emotional self-control over conflicts.I feel I’m a capable person.
6VExpressions of new changes in emotional responding.It doesn’t worry me as much.
6WReferences to self-analysis.It is my lay analysis of this.

In creating the Persian version of the mastery scale, permission was first obtained from Professor Grenyer. Second, forward translation was conducted. Third, back-translation was performed by a bilingual person. Finally, it was reviewed and reconfirmed by two psychologists and the final translation was fixed.

3.3. Statistical Analysis

Data normality was examined by the Shapiro-Wilk test. The inter-rater reliability between three raters was assessed by calculating interclass correlation coefficient ICC (3.3), and test-retest reliability for each rater was assessed by Pearson’s r. Finally, statistical analysis was conducted using SPSS software for windows (version 23).

4. Results

4.1. Test Retest Reliability

A Pearson product moment correlation coefficient was computed to assess the test-retest reliability of the mastery scale. It was assessed by selecting 29 relationship episodes at random and presenting them to raters again after three weeks. A high degree of reliability was found between the first time and the second time for all the raters. The results for raters were as follows: Rater 1’s test-retest reliability r = 0.95, rater 2 r = 0.91, Rater 3 r = 0.80, such that the results were highly significant at P < 0.0001.

4.2. Inter Rater Reliability

To assess the Inter-rater reliability of the mastery scale, three trained and independent raters, scored 36 relationship episodes, then ICC (3, 3) was calculated. Table 2 shows the results of intraclass correlation coefficient (ICC) details. A high degree of reliability was found between the raters. Correlation coefficients between the raters were as follows: rater 1, 2 = 0.89, rater 1, 3 = 0.80, rater 2, 3 = 0.85. The average measure ICC was 0.84 with a 95% confidence interval from 0.72 to 0.90, which is considered a good level of agreement (13). This value indicates that 84% of the variance is true score variance while the remaining 16% reflects error.

Table 2.

Intraclass Correlation Coefficient for Relationship Episodes Ratings

Intraclass Correlation95% Confidence IntervalF Test with True Value
Lower BoundUpper BoundValuedf1df2Sig
Single Measures0.8400.7420.90816.735700.0001
Average Measures0.9400.8960.96716.735700.0001

5. Discussion

In recent years, the usefulness of psychodynamic therapies in the mental health system has been controversial (14-16), especially because of the historical belief that empirical evidence supporting the effectiveness of these treatments is low (2). Therefore, numerous studies have been carried out in recent years on the effectiveness of psychodynamic therapies, and the findings support the effectiveness of this therapy on mild and complex mental health problems (1). Meta-analyses show the impact of psychodynamic therapies on the symptoms of many psychiatric disorders, for example, personality disorders (17), Depression (18, 19), Schizophrenia (20, 21); and even in long-term psychoanalytic therapy (2, 22, 23). But Kazdin (24) believes that no evidence-based explanation can be offered on how and why the intervention of a psychodynamic psychotherapist can result in changes in patients, and more research is needed on the mechanisms of change and variables involved in the effectiveness of the psychodynamic therapy.

On the other hand, psychodynamic therapies have been of interest in Iran and just a few studies have been performed on its effectiveness in Iranian subjects. However, the consequences and effects of a new treatment should be studied to ensure its effectiveness in a country. As explained in the introduction, it is clear that the measures used to assess outcome should be matched with the nature of psychodynamic therapies. To the best of the authors’ knowledge, although a growing number of studies have assessed the efficacy of psychoanalytic therapy in Iran, only a few specialized scales have been translated into Persian and their psychometric properties studied. Yet, no Persian-translated version of the mastery scale or another scale for rating the content of psychotherapy sessions is available in Persian; hence, the present study is the first attempt in this regard. On the other hand, the results revealed that the scale had excellent inter-rater reliability and good test-retest reliability.

The results of this study are generally consistent with previous findings for the English version of the mastery scale. The inter-rater coefficient correlation between 4 raters was reported to be from 0.75 to 0.89 (8) and between 2 raters 0.83 (7) and the test-retest reliability after 3 weeks with 4 raters was reported to be between 0.77 and 0.92 (8).

To use this tool, it is very important for raters to know the rating and the rules of properly preparing transcripts. Prioritizing the surface of speech and its consistency with scale instances, is as important for consideration as the story itself. Hence, it is crucial that raters read the relationship episode completely before rating.

The sample used in this study was collected through psychotherapy sessions and only complete relationship episodes were used; however, this scale can be useful for incomplete episodes or contents collected outside psychotherapy sessions (8). All the episodes were related to therapy sessions having patients diagnosed with major depression without psychotic symptoms and with higher education degrees and average economic and social status. There was no severe disorder in form and process of their thought and its content was uncomplicated. There will probably be less agreement among raters in rating, when it comes to patients with disorganized and symbolic speech or with thought disorder.

However, this scale paves the way for deep research on psychodynamic therapies. Emotional self-regulation and intellectual self-understanding in inter-personal relationships are important mental capacities and useful indices for evaluating the outcomes of psychodynamic therapies (8). The application of mastery scale, especially in case studies, provides accurate observing and following of changes in the ability of controlling and managing emotions and relationships for researchers, during various stages of psychotherapy.

5.1. Conclusion

Following multiple randomized clinical trials on psychodynamic therapies, currently there is no doubt about the empirical support of these treatments. However, the results of clinical studies are not generalizable; meaning that the effectiveness of a psychotherapy method on individuals in a society does not guarantee its usefulness in other communities.

In recent years, Iranian psychotherapists have been more interested in psychodynamic therapies (25). It seems that widespread use of this therapy, closer examination of the mechanisms of changes, and consequences of psychodynamic therapies in Iranian subjects are necessary. This is not possible without proper tools and scales. What highlights the significance of the findings of this study is the necessity of Iranian researchers to access scales which have been developed specifically for psychodynamic studies. The Mastery scale is a comprehensive tool which was developed according to this objective and based on the concept of mastery in psychodynamic theories.

According to the results, the Persian-translated mastery scale is a reliable tool that can be used in assessing the efficacy of psychodynamics therapies.

This study confirmed only the reliability of the mastery scale. The validity of this scale was studied and approved by Grenyer (8); however, the validity of the Persian version has to be approved as well.

5.2. Limitations and Suggestions

The major limitation of this study was the use of a very homogenized sample of relationship episodes from psychiatric patients suffering from major depression disorder. Also, more studies are needed to assess the agreement among raters with narratives of patients suffering from various psychiatric disorders and also the general population.

Although, there are increasing researches on the psychometric properties of the Persian version of psychology scales, there are just a few appropriate Persian translated scales for psychodynamic research. More considerations for these scales will facilitate future psychodynamic efficacy researches in Iran.

Acknowledgements

References

  • 1.

    Brown KM, Tracy DK. Psychoanalytic psychotherapy in contemporary mental health services: Current evidence, future role and challenges. Br J Psychotherapy. 2014;30(2):229-42. https://doi.org/10.1111/bjp.12074.

  • 2.

    Leichsenring F, Rabung S. Long-term psychodynamic psychotherapy in complex mental disorders: Update of a meta-analysis. Br J Psychiatry. 2011;199(1):15-22. [PubMed ID: 21719877]. https://doi.org/10.1192/bjp.bp.110.082776.

  • 3.

    Leichsenring F, Luyten P, Hilsenroth MJ, Abbass A, Barber JP, Keefe JR, et al. Psychodynamic therapy meets evidence-based medicine: A systematic review using updated criteria. Lancet Psychiatry. 2015;2(7):648-60. https://doi.org/10.1016/s2215-0366(15)00155-8.

  • 4.

    Shedler J. The efficacy of psychodynamic psychotherapy. Am Psychol. 2010;65(2):98-109. [PubMed ID: 20141265]. https://doi.org/10.1037/a0018378.

  • 5.

    Lacewing M. Psychodynamic psychotherapy, insight, and therapeutic action. Clin Psychol Sci Pract. 2014;21(2):154-71. https://doi.org/10.1111/cpsp.12065.

  • 6.

    Grenyer BFS. Mastering relationship conflicts: Discoveries in theory, research, and practice. Washington: American Psychological Association; 2002. https://doi.org/10.1037/10419-000.

  • 7.

    Comninos A, Grenyer BFS. The influence of interpersonal factors on the speed of recovery from major depression. Psychotherapy Res. 2007;17(2):230-9. https://doi.org/10.1080/10503300600849140.

  • 8.

    Grenyer B. Mastery through psychotherapy. University of Wollongong; 1996.

  • 9.

    Grenyer BF, Luborsky L. Dynamic change in psychotherapy: Mastery of interpersonal conflicts. J Consult Clin Psychol. 1996;64(2):411-6. [PubMed ID: 8871426].

  • 10.

    Grenyer BFS, Amadei G. Mastery scale I : Manuale di ricerca e di attribuzione di punteggio. Milan, Italy: University of Wollongong and Catholic University; 1994. Italian.

  • 11.

    Kim SH. Construction of levels and categories in grenyer`s mastery scale : The changes in the mastery level of the interpersonal conflict through psychotherapy. Korean J Counseling Psychotherapy. 2005;14(2):301-22. Korean.

  • 12.

    Bonett DG. Sample size requirements for estimating intraclass correlations with desired precision. Stat Med. 2002;21(9):1331-5. [PubMed ID: 12111881]. https://doi.org/10.1002/sim.1108.

  • 13.

    Cicchetti DV. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychol Assess. 1994;6(4):284-90. https://doi.org/10.1037/1040-3590.6.4.284.

  • 14.

    Fonagy P, Lemma A. Does psychoanalysis have a valuable place in modern mental health services? Yes. BMJ. 2012;344:1211. [PubMed ID: 22349582]. https://doi.org/10.1136/bmj.e1211.

  • 15.

    Smit Y, Huibers MJ, Ioannidis JP, van Dyck R, van Tilburg W, Arntz A. The effectiveness of long-term psychoanalytic psychotherapy--a meta-analysis of randomized controlled trials. Clin Psychol Rev. 2012;32(2):81-92. [PubMed ID: 22227111]. https://doi.org/10.1016/j.cpr.2011.11.003.

  • 16.

    Waddingham R. Researching psychodynamic psychotherapy for psychosis: Are we ready yet? Br J Psychotherapy. 2016;32(2):274-8. https://doi.org/10.1111/bjp.12221.

  • 17.

    Town JM, Abbass A, Hardy G. Short-term psychodynamic psychotherapy for personality disorders: A critical review of randomized controlled trials. J Pers Disord. 2011;25(6):723-40. [PubMed ID: 22217220]. https://doi.org/10.1521/pedi.2011.25.6.723.

  • 18.

    Driessen E, Cuijpers P, de Maat SC, Abbass AA, de Jonghe F, Dekker JJ. The efficacy of short-term psychodynamic psychotherapy for depression: A meta-analysis. Clin Psychol Rev. 2010;30(1):25-36. [PubMed ID: 19766369]. https://doi.org/10.1016/j.cpr.2009.08.010.

  • 19.

    Beutel ME, Stark R, Pan H, Silbersweig D, Dietrich S. Changes of brain activation pre- post short-term psychodynamic inpatient psychotherapy: An fmri study of panic disorder patients. Psychiatry Res. 2010;184(2):96-104. [PubMed ID: 20933374]. https://doi.org/10.1016/j.pscychresns.2010.06.005.

  • 20.

    Rosenbaum B, Harder S, Knudsen P, Koster A, Lindhardt A, Lajer M, et al. Supportive psychodynamic psychotherapy versus treatment as usual for first-episode psychosis: Two-year outcome. Psychiatry. 2012;75(4):331-41. [PubMed ID: 23244011]. https://doi.org/10.1521/psyc.2012.75.4.331.

  • 21.

    Harder S, Koester A, Valbak K, Rosenbaum B. Five-year follow-up of supportive psychodynamic psychotherapy in first-episode psychosis: Long-term outcome in social functioning. Psychiatry. 2014;77(2):155-68. [PubMed ID: 24865198]. https://doi.org/10.1521/psyc.2014.77.2.155.

  • 22.

    Leichsenring F, Rabung S. Effectiveness of long-term psychodynamic psychotherapy: A meta-analysis. JAMA. 2008;300(13):1551-65. [PubMed ID: 18827212]. https://doi.org/10.1001/jama.300.13.1551.

  • 23.

    de Maat S, de Jonghe F, Schoevers R, Dekker J. The effectiveness of long-term psychoanalytic therapy: A systematic review of empirical studies. Harv Rev Psychiatry. 2009;17(1):1-23. [PubMed ID: 19205963]. https://doi.org/10.1080/10673220902742476.

  • 24.

    Kazdin AE. Understanding how and why psychotherapy leads to change. Psychother Res. 2009;19(4-5):418-28. [PubMed ID: 19034715]. https://doi.org/10.1080/10503300802448899.

  • 25.

    Shafti SS. Psychoanalysis in Persia. Am J Psychother. 2005;59(4):385-9. [PubMed ID: 16555466].