Translation and Evaluation of the Reliability and Validity of Eating Disorder Inventory -3 Questionnaire Among Iranian University Students

1. Background

Eating disorder inventory (EDI) is a self-reported questionnaire has been used widely for the assessment of symptoms in clinical samples and research. In 1983, the EDI was originally created by Garner, Olmsted, and Polivy (1). It consists of three subscales to measure eating disorder symptoms, such as Bulimia (B), body dissatisfaction (BD), drive to thinness (DT) and also another five psychological domains that are relevant to eating disorders, i.e., ineffectiveness, perfectionism, interpersonal distrust, interceptive awareness and maturity fears. The EDI has been revised for first time in 1991 and its original format was retained while including 27 items to measure further general features such as asceticism, impulse regulation and social insecurity (2, 3).

The EDI-3 is an improved and upgraded version of the earlier EDI, that includes 91 questions (4). The new version of this questionnaire is made up of eating disorder related psychological trait subscales of low self-esteem, personal alienation, interpersonal insecurity, interpersonal alienation, interoceptive deficit , emotional dysregulation, perfectionism, asceticism, maturity fear and three additional response style indices (4). By using EDI-3, index scores gathered from eating disorder patients yielded privileged reliability (4).

Also, the third version of EDI presents acceptable convergent and discriminative validity (5).

In addition, in comparison with EDI-2, scores were recalibrated from 0 - 3 to 0 - 4 format to enlarge the range of cumulative scores for better psychometric traits in normal population (4). Moreover, the suitability of Garner’s model has been evaluated compared with four alternative ones. In the latter of them, the nine primary psychological disturbance factors were separated in two disturbance factors. One representing rigidity and inflexibility (emotional dysregulation, perfectionism, asceticism and interoceptive deficits) and another for representing insecurity and estrangement such as adulthood fear, lack of self-respect, insecurity feeling, personal and interpersonal alienation) (4).

Stanford et al used EDI-3 and the introductory eating disorder assessment for men (EDAM) to explore their repetition in males and the degree of their differentiation among men and women. The results back the theory that eating disorders are considerably different among men and women and support the evidence that developing a valid and reliable eating disorder assessment tool is required specifically in men (6).

Also, Matini et al have compared the intensity and commonness of major depression, general anxiety and eating disorders prior to and after bariatric surgery using several questionnaires including EDI-3 (7).

2. Objectives

The reliability is a characteristic of the scores on a test for a distinctive society of examinees; so reliability coefficients may vary depending on characteristics of the sample. To the best of our knowledge there is no validated version of EDI-3 questionnaire to explore this questionnaire’s validity and reliability between both male and females in Iranian population, especially in high risk people. Because eating disorders commonly present during the teen years or young adulthood, we decided to evaluate university students.

The first aim of our study was to present intra-class correlation coefficients (ICC) to determine the reliability of the questionnaire’s subscales and the second one was to examine the content validity of subscales and also the whole questionnaire by using a large sample of youth population.

3. Methods

3.1. Participants

In this cross sectional study, 480 male and female students were recruited from different universities in Iran. The inclusion criteria were age of 18 years and older (with no upper age limitation) and being university students. To avoid selection bias related to special student groups, samples were selected from different faculties. Participants were recruited since April 2009- September 2010 using convenience sampling method and all of them signed a consent form and the study protocol conforms to the ethical guidelines of the 1975 declaration of Helsinki. The study was accepted by the ethics committee of Tehran University of Medical Sciences with the registration code of (5253-53-01-86). Questionnaires with more than 20% missing data were excluded from the analyses. The EDI-3 is basically a self-administrated questionnaire but to prevent missing questions due to uninformed participants, the examiner (H.D) who administrated the questionnaire, had to visually review the forms to detect unanswered and skipped items or items that had multiple responses to reduce missing items that might invalidate the protocol.

3.2. EDI-3 Questionnaire Content

The EDI-3 questionnaire consists of 91 items and 12 subscales. The main scales are the drive for thinness (7 items), the bulimia (8 items) and body dissatisfaction subscales (10 items), the remaining subscales are: low self-esteem (6 items) emotional dysregulation (8 item), perfectionism (6 items), asceticism (7 items) and interoceptive deficits (9 items) maturity fear (8 items), interpersonal insecurity (7 items), personal alienation (7 items) and interpersonal alienation (7 items). The response options based on a 6-point Likert-type scale are: always, usually, often, sometimes, rarely, and never (8). There are 12 primary scores, six complex scores and three response style validity indices. The primary scores are established on main scales of questionnaire and consist of bulimia, body dissatisfaction, drive for thinness, low self-respect, personal and interpersonal alienation, interpersonal uncertainty, interceptive deficits, asceticism, emotional dysregulation, perfectionism, and maturity fears. The six complex scores are eating disorder risk composite, interpersonal problems composite, ineffectiveness composite and general psychological maladjustment composite. The three response style indices are negative impression, inconsistency and infrequency (9). Scores in each domain range from 0 to 4 (poor, fair, good and excellent).

3.3. Translation Process

First of all, based on the standardized cross-cultural translation guidelines project (9), the English form of the EDI-3 questionnaire was translated into Persian by an experienced Iranian sports medicine physician and psychologist independently. Next, a single questionnaire was made by working and relieving disagreements in two translations by translators and another research team member. Then, the questionnaire was back translated from Persian to English by a physician who was familiar with psychological terminology. He also, had lived and worked in the UK for years and was not familiar with the EDI-3 questionnaire. After checking and discussion, both researchers and translators reached agreement on the final Persian version. Eventually a pilot study was conducted using the new questionnaire on 23 volunteered participants to determine the content validity of the questionnaire.

3.4. Validity Assessment

The first phase of questionnaire development was the selection of a panel of experts. They were composed of one sports medicine specialist, one psychiatrist, and one clinical epidemiologist. They evaluated the validity of each query as well as the entire questionnaire. Two indexes (relevancy and clarity) were evaluated for each query, and four indices (relevancy, clarity, comprehensiveness and inter-rater agreement) were computed for the whole questionnaire. For each question and also the whole questionnaire, experts were asked to score from 1 to 4, “a four-point scale of poor, fair, good and excellent”, respectively. To analyze the validity index, the scores were considered as two groups: good or excellent vs. fair or poor. Each items’ validity was calculated as the percentage of specialists rating the question as good or excellent. For each item, cut-off point of 0.80 was considered as acceptable validity. Each query with validity indexes less than 0.80 was revised or excluded (10). Face validity was ensured through discussion between one of expert (HD) and 23 participants. Also three experts (H.D) (N.CH) and one epidemiologist had a meeting to determine face validity of the questionnaire via assessment and evaluation of each item.

3.4.1. Item Generation

The initial concern of the panel of experts and the information gathered from the discussion groups led us to define the proportions considered to be relevant for inclusion in the questionnaire via the generation of a comprehensive list with affirmation format, to reflect as far as possible the expressions directly yielded from the participants in the discussion group.

3.5. Reliability Assessment

The EDI-3 was administered to participants and the importance of full completion of questionnaires was explained clearly. Intra-cass correlation coefficients was used to determine the reliability of the EDI-3 questionnaire in a randomized sample of healthy individuals (ICC -the two-way random model). The ICC is an estimation of the variation among individuals (11). Internal inconsistency was measured using Cronbach’s alpha coefficient. For evaluation of the test-retest reliability, the examiner (H.D) administered the questionnaire to 480 university students for the first time. 150 of them were selected and they completed the questionnaire after 2 weeks for the second time.

3.6. Statistical Analysis

The mean values of basical characteristics such as age, marital status, height and weight were calculated for men and women, separately. Also, the t-test was performed to find out if there was any significant difference between these two groups. Moreover, intra-class correlation coefficients (ICC -the two-way random model) was used to measure test-re-test reliability. To estimate internal consistency for each subscale Cronbach’s alpha was used. An alpha greater than or equal to 0.6 was considered satisfactory (12).

All statistical analysis was carried out using SPSS software (IBM Corp. Released 2012. IBM SPSS statistics for windows, version 21.0. Armonk, NY: IBM Corp).

4. Results

4.1. Study Subjects

Among students who received the questionnaires, 452 filled out the questionnaire completely, and analysis was restricted to them. The physical characteristics are presented in Table 1. There was no significant difference between men and women in age; but the majority of male participants were single and also had higher height and weight compared to women and the difference was statistically significant (All Ps < 0.05).

4.2. Reliability Analysis

As Table 2 shows, interpersonal alienation and asceticism did not achieve an acceptable level of ICC. Cronbach’s Alpha coefficient of both was 0.4. In test-retest analysis, the ICCs for measuring the three subscales of eating disorder were within the range of acceptable values for men and women (DT = 0.91 and 0.88 B = 0.77 and 0.86 and BD = 0.69 and 0.71 respectively).

4.3. Content Validity

The content validity for clarity was acquired higher than 0.8 (89 items out of 91) and for relevancy was also 0.80 or higher (87 out of 91) questions. The queries receiving a score below 0.80 for either clarity (two questions) or relevancy (four questions) were assessed and discussed once more by specialists and completed based on consensus.

The total validity index score for clarity and relevancy were 0.91 and 0.89, respectively.

Based on all three experts, the comprehensiveness of the survey was good or excellent, and total comprehensiveness score of survey was 100%.

5. Discussion

Our study’s results suggest that the Persian version of EDI-3 would be a reliable questionnaire with good validity for measurement of eating disorders in healthy Persian adults. Also, the study findings demonstrated acceptable alpha coefficients in all domains but asceticism and interpersonal alienation subscales in both sexes. In general, the results of this study are partly in agreement with other researches that have reported that EDI-3 is a valid and reliable tool on normative samples from the USA, Canada, Europe and Australia (12). Also, Nyman-Carlsson et al., stated acceptable reliability for all subscales of EDI-3 questionnaire except asceticism among Swedish normal controls (13). In addition, Tachikawa et al, administered the Japanese version of eating disorder inventory (EDI-2) to 91 Japanese eating disordered inpatients and 119 matched non-clinical controls. The results of their study showed that all subscales (except asceticism) reached sufficient internal consistencies (14). It can be attributed to the unknown feeling that some respondents declared. Also, it might be that these domains do not appear very homogenous at least in adult Iranian culture. The other possible explanation would be that the responders do not have a very strong tendency to place a positive connotation on achieving virtue through self-restrain or to experience guilt and shame surrounding the experience of pleasure. A low score scale not only is common among respondents from the nonclinical sample but also could reflect denial of current clinical state or response bias on the part of the patients.

Drive to thinness is one of the main signs and also is considered as the essential criterion for diagnosis of eating disorder. Seven questions related to this subscale evaluate these items: desire to be thin, worry about dieting, preoccupation with weight and intense fear of weight gain. The ICC value for drive to thinness was 0.8 in our study, but Clausen et al., reported that drive to thinness may score high for any reason unrelated to the pathology of eating disorders or inaccuracy in perception or recognition of bodily states so this important pathognomonic sign of the specific eating disorder psychopathology, frequently led to missteps in distinguishing signs of hunger (4).

The bulimia scale measures the issue of binge eating and only one of its items referred to the compensatory method of vomiting. Studying 108 American men and women, Stanford et al. showed that both the male and female subjects’ scores were mainly similar for eating disorders, reflecting a diagnosis of binge eating. As he stated, which is in line with our study, the bulimia scale on the EDI-3 in point of fact, sufficiently evaluates binge eating symptoms in men (6).

Perfectionism subscale includes six items to evaluate respondent’s desire for hard work to achieve high goals. Questions related to the subscale of perfectionism are divided into two dimensional constructs; self-oriented perfectionism and perfectionism is reflected by the community over the individual (15, 16). This subscale showed a borderline score in our study. This could be due to these two independent dimensional constructions of items related to this subscale. For instance, one might be perfectionism, but this feeling is not related to the expectations of those around a person. Furthermore, a person may be under pressure from expectations of others indicating in this item “my parents have expected excellence of me” ,but the best in his life is not one’s own effort relating in this item “I hate being less than best at things”. It seems that if perfectionism is divided into two categories and evaluated separately, the result would be more satisfying. However, testing such hypotheses is beyond the scope of this study.

The ICC values in current study for body dissatisfaction in men and women were 0.6 and 0.8 respectively, along with the Stanford et al study, asserted that the body dissatisfaction subscale alone could not remarkably aid the recognition of an eating disorder in males (6). However, it could be said that the Stanford et al study participants were 66 men who received treatment for eating disorder and addiction and 45 healthy ones were matched as control group (6). The comparison between Swedish, Danish and international control samples demonstrated that the Swedish control scored notably higher than the other two controls, unless in the eating disorder risk scales, perfectionism and asceticism (13). While Swedish controls scored higher on interoceptive deficits, emotional dysregulation and maturity fears, the international counterpart achieved higher score on interpersonal alienation subscale (13). Also, Nyman-Carlsson et al reported that psychiatric outpatients scored higher than both ED and normal to controls just in two scales, including interpersonal insecurity and interpersonal alienation (13). Furthermore, a study released by Waldherr et al, showed that the average EDI scores of the southern countries are similar to the North American and tendency to weight seems to be less prevalent in the Netherlands than in North America and central and southern parts of Europe. Also, due to the subscales of bulimia, perfectionism, interoceptive awareness and maturity fears, the Dutch females of all age groups scored lower than Italian females. In addition, Italians earned slightly higher scores in bulimia and interoceptive awareness than the US. In sum, the results of the Waldherr et al study revealed that the EDI scores may be affected by socio-cultural factors, cultural properties, culture-specific social requirements on young adults and differences in maturation between North and South Europe (3).

The Mahmoodi et al study’s findings indicated that although restraint, eating concerns, laxative misuse and self-induced vomiting are rare phenomena in Iranian undergraduate women, but shape concern, objective binge eating, and dietary restraint are commonly seen.

Also, this study presented some similarities between Iranian and Spanish students on the scores of the eating, weight and shape concerns subscales. Furthermore, Iranian and Portuguese women were similar on shape concern subscale. But Iranian women obtained a higher score on restraint subscale than Japanese female students (17).

In a review study that ED assessments in athletes have been observed, results showed that 24 studies calculated internal consistency and only three of them calculated test-retest reliability. In 14 studies, the second version of EDI has been used and most of them have cited Garner (1983, 1991) reliability coefficients except three that calculated coefficients ranged between 0.69-0.90. The writer has proposed that the other types of reliability assessments including test-retest reliability would be valuable to evaluate whether or not athletes may earn approximately the same EDs score during multiple measurements and /or make sure that changes of EDs scores overtime are not due to measurement property change of eating disorder assessments (18).

The uses of EDI-3 vary from diagnostic perception, therapy plans, and outcomes interpretation for female eating disorder patients to ED screening in nonclinical groups. Overall, considering these parameters, such as considerably expanded patient samples, upgraded scale derivation methods and theoretical structure, inclusion of response style indices, and a better section on test analysis, the new EDI-3 seems to be experimentally superior to EDI-2. But it was designed for use with females aged 13 - 53 years, and then a lack of information about its value and appeal with men in both clinical and nonclinical settings is a primary deficit (5).

One of the strengths of our study is the careful planning and proper execution of the test-retest reliability. A comprehensive number of university students were registered and encouraged to fill out questionnaires as completely as possible. Moreover, the time and place was arranged precisely in the same place to carry out the re-test two weeks later.

5.1. Limitations

The results of this study need interpretation with caution. First of all, it could be said that the sampling method and nature of the sample (i.e., only university students) limit the generalization of the findings into the general population and clinical sample.

Secondly, the university students should fill out a large questionnaire. In some situations, they did not complete the questionnaire completely. We prepared refreshments for them and asked to fill out the whole questionnaire completely by a gifted pencil. The examiner (H.D) should check all questionnaires to ensure that all items were answered. But some items were missed in spite of precise assessment. Finally, it's worth noting that calculating discriminant or convergent validity especially in the patients group would be much more preferable.

In conclusion, based on our study results, the validity and reliability of Persian version of EDI-3 is acceptable in all subscales except asceticism and interpersonal alienation in both men and women.