Cross-cultural Adaptation, Validity, and Reliability of the Persian Utrecht Scale for Evaluation of Rehabilitation-Participation in Spinal Cord Injury Patients

authors:

avatar Amir Javanmard ORCID 1 , avatar Kianoush Abdi ORCID 1 , * , avatar Abbas Ebadi ORCID 2 , avatar Samaneh Hosseinzadeh ORCID 3

Department of Rehabilitation Management, University of Social Welfare and Rehabilitation Sciences, Tehran, Iran
Behavioral Sciences Research Center, Life Style Institute, Baqiyatallah University of Medical Sciences, Tehran, Iran
Department of Biostatistics, University of Social Welfare and Rehabilitation Sciences, Tehran, Iran

how to cite: Javanmard A, Abdi K, Ebadi A, Hosseinzadeh S. Cross-cultural Adaptation, Validity, and Reliability of the Persian Utrecht Scale for Evaluation of Rehabilitation-Participation in Spinal Cord Injury Patients. Middle East J Rehabil Health Stud. 2021;8(2):e112089. https://doi.org/10.5812/mejrh.112089.

Abstract

Background:

Despite participation is an important outcome of person’s rehabilitation process.there is no reliable and valid instrument for evaluation of participation in Persian-speaking spinal cord injuries (SCI). While SCI has serious damages on person’s social participation, assessment of participation demands culturally adapted and valid instruments.

Objectives:

This study took place to obtain a valid, reliable instrument for evaluation of Rehabilitation-Participation (P-USER-P) in an Iranian SCI.

Methods:

This study was methodological research, participating in 200 individuals. Analysis of face and content validity, discriminant validity, internal consistency, and reliability conducted after receiving permission and verification of the backwards-translation from the main developer.

Results:

During the cultural adaptation process, several modifications proposed by experts’ panel were made to ensure adaptation of items’ descriptions with Iranian culture. The Cronbach’s α and ICC coefficients of the frequency, restriction and satisfaction sub-scales were 0.61, 0.71, 0.69, and 0.75, 0.84, 0.84, respectively. The instrument also showed good reliability and discriminant validity characteristics.

Conclusions:

P-USER-P is a valid and reliable instrument for measuring objective and subjective participation in SCI patients. It is also adapted with Iranian culture which qualifies its utility in the rehabilitation process of Persian-speaking SCI patients.

1. Background

Participation is emerging as an important novel outcome of rehabilitation as world health organization is focusing more precisely on the performance of individuals in their daily activities (1). It is important to evaluate the person’s experienced participation from the initiation of care-delivery through the final steps of rehabilitation (2). The novel concept of “participation” has been defined as an individual’s own experiences of involvement in a life situation and has replaced the concept “handicap” (1). Therefore, it serves as a major outcome of rehabilitation services (3, 4).With regards to a detailed review of current instruments in the field (5), there are few measures of participation in accordance with the concept provided by World Health Organization’s International classification of functioning (6, 7). Therefore, the ultimate goal of rehabilitation in community-based and outpatient settings is directing toward the enhancement of a person’s participation in meaningful activities and important roles of their life (8). Despite this fact, there are a few measures that include both subjective and objective aspects of participation (9-11). The Utrecht Scale for evaluation of rehabilitation-participation is one of the few instruments that intends to assess subjective, as well as the objective status of person’s participation (8). The original version of USER-P has been validated in Switzerland by a cross-sectional survey and all of its sub-scales have shown good Internal consistency (8). The instrument is a reliable and validated measure of participation in rehabilitation (11) and had not been translated into the Persian language before.

Among all physical disabilities, SCI is one of the conditions that would restrict participation in different domains of life (12, 13), and therefore, attaches the individual to a caregiver who should provide him with emotional, physical, and financial care and help him with the tasks of activities of daily living (14). To ensure the enhancement of their independence, the population of SCI persons need to get evaluated in terms of participation. USER-P can help in this manner by verifying the effectiveness of rehabilitation services.

2. Objectives

This study took place in order to verify the Persian version of USER-P in terms of cultural adaptation, validity and reliability for application in Persian-speaking individuals.

3. Methods

3.1. Translation and Cultural-Adaptation

After the main developer accepted the request of translation, IQOLA (International Quality of Life Assessment) method used as a reference for translating the instrument to the Persian language. Initially, the instrument translated to Persian by two experts, one fluent with native English language and the other a specialist in rehabilitation services for persons with SCI. In the next step, one of the authors rated the quality of translation (0 - 100) and reproduced the final revision of translations as a final version besides two experts. The final version of the scale translated from Persian into English and has been sent to the main developer in order to maintain his approval and further studies took place after his agreement on the translated instrument (Appendix 1 in Supplementary File).

3.2. Participants

The sample size of the population selected according to the rule of thumb principle, which indicates five samples for every item in the questionnaire, and it should not be less than 100 samples, according to Gorsuch (15). The final sample consisted of 50 healthy individuals and 150 persons with SCI. The healthy individuals selected from available individuals aged from 20 to 65, with different genders and social and academic levels. In this study, the sampling method was simple, non-random and available. One of the authors administered the completion of the scale by participants, in order to ensure the comprehensiveness of items. Inclusion criteria included having at least 18 years old, the ability of reading and writing in Persian, the cognitive ability to handle questions, and their consent for participating in the study.

The informed consent was obtained from the participants via clarifying the purpose, risks, benefits and the process of the study. The authors confirm they have appropriately handled confidentiality and data security.

3.3. Instruments

USER-P consists of 31 items divided into three sub-scales (Frequency, Restriction, Satisfaction). The frequency sub-scale consists of two parts; Part A determines the number of hours engaged in four general activities during a typical week of individual’s life, and Part B asks the number of times engaged in eight social and leisure activities, such as going out, visiting family and friends, and indoor hobbies in the past four weeks.

In Part A of the frequency sub-scale, the scores range indicate 0 (not at all) to 5 (36 hours or more); and in Part B, from 0 (not at all) to 5 (19 times or more).

The restriction sub-scale consists of 10 items which indicate the experienced participation restrictions as a result of the injury or disability, in the social, vocational and leisure activities. Each item of the restriction sub-scale ranges from 0 (not possible at all) to 3 (independent without difficulty). There’s a “not applicable” answer for each item in case of being irrelative to the person’s situation or if the experienced restriction does not relate to the person’s injury or disability. The satisfaction sub-scale consists of 9 items which indicate the satisfaction with social relationships, vocational activities and leisure. Items of the satisfaction sub-scale range from 0 (very dissatisfied) to 4 (very satisfied). The total score of each sub-scale calculates by the items that were applicable to the participant’s context and situation; the sum score for each sub-scale should be converted to a 0 - 100 scale. Higher scores demonstrate higher levels of participation. There is no total score for the USER-Participation (11).

Data was collected from April to September 2019, and analysis study took place afterwards until December 2019.

3.4. Validity and Reliability Assessment

The process of validity and reliability took out through the following steps: Face validity (cognitive interview), content validity (qualitative), Discriminant validity, and then for reliability, we used internal consistency (Cronbach’s alpha) and test-retest by intraclass correlation coefficients (ICC).

Face validity conducted by partaking 10 SCI persons with elementary education. Initial assessment guided the authors to re-evaluate the items based on the understanding of the target population. In the next step, qualitative content validity method conducted by a panel of 10 experts including English-fluent rehabilitation and healthcare professors, experienced with the adaptation of evaluation instruments. The panel assessed the content validity of the scale by reviewing the items in terms of relevancy of each item to the domains of participation based on content validity procedure. Discriminant validity performed between healthy and SCI individuals, whilst two-independent samples t-test used. In order to verify the reliability and validity of the native instrument and applicability to the target population, assessment of the psychometric properties took place. Cronbach’s alpha used for assessing internal consistency.Paired T-tests were used for test-retest analysis, and ICC implied for assessing the stability of scores over time. To assess reproducibility, 30 individuals from the sample retook the scale after two weeks from the initial administration. Test-retest reliability evaluated by using ICC between the initial and second assessment. Values less than 0.5, between 0.5 and 0.75, between 0.75 and 0.9, and greater than 0.90 were indicative of poor, moderate, good, and excellent reliability, respectively (16). The socio-demographic information gathered using an additional one-page form. Statistical analysis conducted by Statistical Package for the Social Sciences (SPSS) Version 22.

4. Results

4.1. Participant Characteristics

Most of the population were male (59.7%). One-third of the participants did not continue higher education after diploma (33.3%), and 18.7% of them did not graduate from high school. In terms of insurance, about half of the participants were covered by basic health insurance, and more than half (62.7%) of them were provided by the services of social welfare organization. Full demonstration of participants’ characteristics has provided in Table 1.

Table 1.

Characteristics of Total Participants in the Study (150 Persons)

VariablesValues a
Gender
Male58 (38.7)
Female92 (61.3)
Education
High school or below28 (18.7)
Diploma50 (33.3)
Associate degree24 (16)
Bachelors40 (26.7)
Masters8 (5.3)
Marital status
Single39 (26)
Married89 (59.3)
Divorced12 (8)
Widowed10 (6.7)
Occupation
Employed66 (44)
Unemployed84 (56)
Injury type
Diplegia58 (38.7)
Paraplegia54 (36)
Quadrplegia12 (8)
Triplegia11 (7.3)
Paraparesia15 (10)

4.2. Translation and Cultural Adaptation

The cultural adaptation included forward translation, evaluation of the translation, and backward translation, consecutively.In the evaluation process, the word “partner” in the items that asked the participants’ relationship with his/her partner, were culturally odd for the ordinary individual, as the courtship only applies in the form of legal marriage in Iranian society; therefore, the word “partner” replaced with the word “spouse”, and then translated into the Persian language to be more understandable for the Iranian participant.

4.3. Evaluation of Psychometric Characteristics

Psychometric characteristics include face and content validity; discriminant validity, reliability that has addressed in separate parts.

4.3.1. Face Validity

In terms of face validity, initial participators which consisted of ten SCI patients with elementary education assigned the fluency and relevancy of the items. The authors then re-evaluated the initial translated version and sent the revised version to the expert panel in order to assess content validity.

4.3.2. Content Validity

In order to enrich the cultural adaptation process, several modifications were made to enhance the fluency of scale, especially for those participants who have had elementary education. For instance, several descriptions of the items were modified with respect to the context of the Iranian participant. Outdoor activities such as going to the beach, concert and/or attending church were replaced by more common activities in Iranian culture such as going to the mosque and attending public social events (Table 2).

Table 2.

Expert Panel Suggestions on Qualitative Content Validity

NumberItemPanel RecommendationFinal Correction
1Sport (such as: Tennis, Bicycling, and gym)Examples of this item should adapt for a typical spinal cord injury personMain examples replaced by Sitting volleyball and cycling with wheelchair
2Going Out (e.g. Coffee shop, concert, theatre)It is recommended to replace the examples of this item with more identical activities in Iranian cultureMentioned activities have been replaced by cultural events and art performances.
3Your relationship with your partner (such as: communication, sexuality)“Communication” needs to have a more common equivalent in Persian languageThe translation of “communication” has been established as equivalent of “reciprocal relation” in Persian language
4Attending events, going to the beach, church or mosqueSocial events that are more common in Iranian culture should replace the mentioned activitiesAttending private gatherings and public events, going to mosque and other religious spots replaced for mentioned examples of going out

The expert committee proposed several modifications to adapt the items with native culture.

4.3.3. Discriminant Validity

In terms of discriminant validity, the difference between groups that are certainly distinct in terms of the questionnaire’s construct get compared (17). Subsequently, scores of 50 normal individuals, got compared with 150 SCI persons who had completed P-USER-P (18). According to the P-value which is smaller than 0.001, all three sub-scales had meaningful discriminant coefficients. (Table 3).

Table 3.

Compare Sub-scales Scores Between Normal-Range and Persons with Spinal Cord Injury (SCI)

Sub-scaleNMean ± SDtP-Value
Frequency-6.051< 0.001
Normal15027.35 ± 4.216
SCI5032.98 ± 6.116
Restriction-11.717< 0.001
Normal15030.65 ± 4.685
SCI5040.14 ± 5.721
Satisfaction-12.443< 0.001
Normal15027.29 ± 4.426
SCI5036.28 ± 4.426

4.3.4. Reliability

Internal consistency of the P-USER-P was assessed using Cronbach’s alpha. The ICC and cronbach’s alpha values more than 0.7 indicated moderate and good internal consistency in different sub-scales (19). Table 4 demonstrates the results of internal consistency for each sub-scale.

Table 4.

Internal Consistency of P-USER-P Sub-scales

ScaleNumber of ItemsCronbach’s Alpha
Frequency110.615
Restrictions110.715
Satisfaction90.695

The reproducibility of the P-USER-P also verified by 30 SCI persons after two weeks from the initial assessment. In this study, coefficients of internal consistency and test-retest reliability showed generally moderate to very good results. Results of the T-test (Table 5) showed good ICC score in P-USER-P frequency sub-scale (0.755), and very good ICC scores in restrictions (0.849) and satisfaction (0.846) sub-scales between the scores of test and retest.

Table 5.

Test-Retest Reliability Scores of P-USER-P Sub-scales (N = 30)

Sub-scalesTest, Mean ± SDRetest, Mean ± SDAbsolute ICCUpper BoundLower BoundP-Value
Frequency27.35 ± 4.2127.23 ± 3.070.6060.3200.7910.001
Restriction30.66 ± 4.6928.2 ± 4.490.7370.5170.8660.001
Satisfaction27.29 ± 4.4228.5 ± 4.020.7340.5120.8640.001

5. Discussion

This study aimed at assessing the cross-cultural adaptation of the USER-P to Persian to develop the P-USER-P and its reliability and validity. While there are few scales that aim to evaluate both subjective and objective participation of rehabilitation patients, P-USER-P counts as a valid reliable instrument that has been translated and validated in at least one Asian language before Persian and has been proven to be congruent with Eastern cultures (20).The USER-P had been also translated into the Korean language and tested in terms of psychometric properties with a sample of stroke patients by Lee et al. (20). They found three subscales of K-USER-P to have Cronbach's alpha between 0.66 and 0.69. Also, in terms of test-retest reliability, they found that the satisfaction scale had 0.71 ICC, and frequency and restriction subscale had 0.63 and 0.45, respectively. These results are compatible with the findings of our study, and also, higher ICC scores indicate that P-USER-P is a more reliable tool than K-USER-P. The results of the reliability study were compatible with the psychometric study of original USER-P which was conducted by Mader et al. on a population of rehabilitation outpatients in The Netherlands (8). Their research showed Cronbach’s alpha of 0.65 for the Frequency, as well as 0.90 for both satisfaction and restrictions subscales. The only advantage of Mader et al.’s research was their higher value of alpha for Restriction and Satisfaction scales which were about 0.2 higher (8). Another psychometric study of original USER-P by Post et al. found that internal consistency of the Restrictions and Satisfaction scales was very good, alphas were high and all item–rest correlations well above the criterion (11). Similarly, we found the similar values for Restriction and Satisfaction subscales.

Face and content validity performed by applying minor changes in translation to increase the comprehensiveness of items within the cultural context of Iran.

In terms of discriminant validity, our research showed that all subscales of P-USER-P distinguish between normal-range and SCI adults. Van der Zee et al. (21) in their discriminant validity examination concluded that USER-P demonstrated significant differences in levels of participation between persons with different health conditions and different levels of functional limitations, overall, the Restrictions score was sensitive to variations in their research (5). In another relevant study, participation measure for post-acute care (PM-PAC) questionnaire, showed generally varied subscale scores on the concept of condition severity and the ability for mobility (22). Overall, the initial goals of the research obtained.

The strengths of this study are its unique and comprehensive thoroughness in following the cultural and linguistic adaptation, and the findings of the authors and translators at reaching a consensus version. Furthermore, the final version was dedicated to a Persian-speaking SCI population. The limitations of this study include the lack of a Persian instrument similar to USER-P for assessing the concurrent validity, therefore this investigation is highly recommended in the future researches. Also, the instalment of a proper online communication pathway for accessing the persons with SCI would enhance the productivity of the future researches.

5.1. Conclusions

Community administrators and rehabilitation specialists should adapt their health and wellbeing plans based on indicators that enable the individuals to attempt in beneficial activities that can support a person’s livelihood and the dynamic economy. Results of this study are important because the P-USER-P can be used by occupational therapists and other rehabilitation specialists to measure the level of participation in persons with SCI in Iran. The findings of this study approves P-USER-P as a valid and reliable instrument for evaluation of Rehabilitation-Participation in an Iranian SCI population.

Acknowledgements

References

  • 1.

    World Health Organization. International classification of functioning, disability and health: children and youth version: ICF-CY. WHO; 2007. Available from: https://apps.who.int/iris/handle/10665/43737.

  • 2.

    Piskur B, Daniels R, Jongmans MJ, Ketelaar M, Smeets RJ, Norton M, et al. Participation and social participation: are they distinct concepts? Clin Rehabil. 2014;28(3):211-20. [PubMed ID: 23988324]. https://doi.org/10.1177/0269215513499029.

  • 3.

    Wolf AC, Tate RL, Lannin NA, Middleton J, Lane-Brown A, Cameron ID. The World Health Organization disability assessment scale, WHODAS II: reliability and validity in the measurement of activity and participation in a spinal cord injury population. J Rehabil Med. 2012;44(9):747-55. [PubMed ID: 22854805]. https://doi.org/10.2340/16501977-1016.

  • 4.

    World Health Organization. World report on disability 2011. WHO; 2011. Available from: https://www.who.int/teams/noncommunicable-diseases/sensory-functions-disability-and-rehabilitation/world-report-on-disability.

  • 5.

    Javanmard A, Abdi K, Ebadi A, Hosseinzadeh S. Participation instruments in persons with spinal cord injury: A narrative review. Med J Islam Repub Iran. 2020;34:66. [PubMed ID: 32974232]. [PubMed Central ID: PMC7500424]. https://doi.org/10.34171/mjiri.34.66.

  • 6.

    Cardol M, Brandsma JW, de Groot IJ, van den Bos GA, de Haan RJ, de Jong BA. Handicap questionnaires: what do they assess? Disabil Rehabil. 1999;21(3):97-105. [PubMed ID: 10206348]. https://doi.org/10.1080/096382899297819.

  • 7.

    Van De Velde D, Bracke P, Van Hove G, Josephsson S, Viaene A, De Boever E, et al. Measuring participation when combining subjective and objective variables: the development of the Ghent Participation Scale (GPS). Eur J Phys Rehabil Med. 2016;52(4):527-40.

  • 8.

    Mader L, Post MW, Ballert CS, Michel G, Stucki G, Brinkhof MW. Metric properties of the Utrecht Scale for Evaluation of Rehabilitation-Participation (USER-Participation) in persons with spinal cord injury living in Switzerland. J Rehabil Med. 2016;48(2):165-74. [PubMed ID: 26926920]. https://doi.org/10.2340/16501977-2010.

  • 9.

    Noonan VK, Kopec JA, Noreau L, Singer J, Masse LC, Zhang H, et al. Comparing the validity of five participation instruments in persons with spinal conditions. J Rehabil Med. 2010;42(8):724-34. [PubMed ID: 20809054]. https://doi.org/10.2340/16501977-0584.

  • 10.

    Noonan VK, Miller WC, Noreau L, Scire Research Team. A review of instruments assessing participation in persons with spinal cord injury. Spinal Cord. 2009;47(6):435-46. [PubMed ID: 19238163]. [PubMed Central ID: PMC3535488]. https://doi.org/10.1038/sc.2008.171.

  • 11.

    Post MW, van der Zee CH, Hennink J, Schafrat CG, Visser-Meily JM, van Berlekom SB. Validity of the utrecht scale for evaluation of rehabilitation-participation. Disabil Rehabil. 2012;34(6):478-85. [PubMed ID: 21978031]. https://doi.org/10.3109/09638288.2011.608148.

  • 12.

    Chiodo AE, Scelza WM, Kirshblum SC, Wuermser LA, Ho CH, Priebe MM. Spinal cord injury medicine. 5. Long-term medical issues and health maintenance. Arch Phys Med Rehabil. 2007;88(3 Suppl 1):S76-83. [PubMed ID: 17321853]. https://doi.org/10.1016/j.apmr.2006.12.015.

  • 13.

    Noreau L, Fougeyrollas P, Post M, Asano M. Participation after spinal cord injury: the evolution of conceptualization and measurement. J Neurol Phys Ther. 2005;29(3):147-56. [PubMed ID: 16398947]. https://doi.org/10.1097/01.npt.0000282247.15911.dc.

  • 14.

    Farajzadeh A, Akbarfahimi M, Maroufizadeh S, Rostami HR, Kohan AH. Psychometric properties of Persian version of the Caregiver Burden Scale in Iranian caregivers of patients with spinal cord injury. Disabil Rehabil. 2018;40(3):367-72. [PubMed ID: 28637138]. https://doi.org/10.1080/09638288.2016.1258738.

  • 15.

    Gorsuch RL. Factor analysis: Classic edition. 2nd ed. New York: Routledge; 2014.

  • 16.

    Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155-63. [PubMed ID: 27330520]. [PubMed Central ID: PMC4913118]. https://doi.org/10.1016/j.jcm.2016.02.012.

  • 17.

    Lucas RE, Diener E, Suh E. Discriminant validity of well-being measures. J Pers Soc Psychol. 1996;71(3):616-28. [PubMed ID: 8831165]. https://doi.org/10.1037//0022-3514.71.3.616.

  • 18.

    Hattie J, Cooksey RW. Procedures for assessing the validities of tests using the "known-groups" method. Appl Psychol Meas. 2016;8(3):295-305. https://doi.org/10.1177/014662168400800306.

  • 19.

    Polit DF, Yang F. Measurement and the measurement of change: a primer for the health professions. Philadelphia, PA: Wolters Kluwer; 2016.

  • 20.

    Lee JH, Park JH, Kim YJ, Lee SH, Post MWM, Park HY. Validity and reliability of the Korean version of the Utrecht scale for evaluation of rehabilitation-participation. Occup Ther Int. 2017;2017:9452051. [PubMed ID: 29097981]. [PubMed Central ID: PMC5612671]. https://doi.org/10.1155/2017/9452051.

  • 21.

    van der Zee CH, Post MW, Brinkhof MW, Wagenaar RC. Comparison of the Utrecht scale for evaluation of rehabilitation-participation with the ICF measure of participation and activities screener and the WHO disability assessment Schedule II in persons with spinal cord injury. Arch Phys Med Rehabil. 2014;95(1):87-93. [PubMed ID: 24008052]. https://doi.org/10.1016/j.apmr.2013.08.236.

  • 22.

    Gandek B, Sinclair SJ, Jette AM, Ware JJ. Development and initial psychometric evaluation of the participation measure for post-acute care (PM-PAC). Am J Phys Med Rehabil. 2007;86(1):57-71. [PubMed ID: 17033591]. https://doi.org/10.1097/01.phm.0000233200.43822.21.