Abstract
Background:
Despite participation is an important outcome of person’s rehabilitation process.there is no reliable and valid instrument for evaluation of participation in Persian-speaking spinal cord injuries (SCI). While SCI has serious damages on person’s social participation, assessment of participation demands culturally adapted and valid instruments.Objectives:
This study took place to obtain a valid, reliable instrument for evaluation of Rehabilitation-Participation (P-USER-P) in an Iranian SCI.Methods:
This study was methodological research, participating in 200 individuals. Analysis of face and content validity, discriminant validity, internal consistency, and reliability conducted after receiving permission and verification of the backwards-translation from the main developer.Results:
During the cultural adaptation process, several modifications proposed by experts’ panel were made to ensure adaptation of items’ descriptions with Iranian culture. The Cronbach’s α and ICC coefficients of the frequency, restriction and satisfaction sub-scales were 0.61, 0.71, 0.69, and 0.75, 0.84, 0.84, respectively. The instrument also showed good reliability and discriminant validity characteristics.Conclusions:
P-USER-P is a valid and reliable instrument for measuring objective and subjective participation in SCI patients. It is also adapted with Iranian culture which qualifies its utility in the rehabilitation process of Persian-speaking SCI patients.Keywords
Quality of Life Spinal Cord Injury Validity and Reliability Functional Disability
1. Background
Participation is emerging as an important novel outcome of rehabilitation as world health organization is focusing more precisely on the performance of individuals in their daily activities (1). It is important to evaluate the person’s experienced participation from the initiation of care-delivery through the final steps of rehabilitation (2). The novel concept of “participation” has been defined as an individual’s own experiences of involvement in a life situation and has replaced the concept “handicap” (1). Therefore, it serves as a major outcome of rehabilitation services (3, 4).With regards to a detailed review of current instruments in the field (5), there are few measures of participation in accordance with the concept provided by World Health Organization’s International classification of functioning (6, 7). Therefore, the ultimate goal of rehabilitation in community-based and outpatient settings is directing toward the enhancement of a person’s participation in meaningful activities and important roles of their life (8). Despite this fact, there are a few measures that include both subjective and objective aspects of participation (9-11). The Utrecht Scale for evaluation of rehabilitation-participation is one of the few instruments that intends to assess subjective, as well as the objective status of person’s participation (8). The original version of USER-P has been validated in Switzerland by a cross-sectional survey and all of its sub-scales have shown good Internal consistency (8). The instrument is a reliable and validated measure of participation in rehabilitation (11) and had not been translated into the Persian language before.
Among all physical disabilities, SCI is one of the conditions that would restrict participation in different domains of life (12, 13), and therefore, attaches the individual to a caregiver who should provide him with emotional, physical, and financial care and help him with the tasks of activities of daily living (14). To ensure the enhancement of their independence, the population of SCI persons need to get evaluated in terms of participation. USER-P can help in this manner by verifying the effectiveness of rehabilitation services.
2. Objectives
This study took place in order to verify the Persian version of USER-P in terms of cultural adaptation, validity and reliability for application in Persian-speaking individuals.
3. Methods
3.1. Translation and Cultural-Adaptation
After the main developer accepted the request of translation, IQOLA (International Quality of Life Assessment) method used as a reference for translating the instrument to the Persian language. Initially, the instrument translated to Persian by two experts, one fluent with native English language and the other a specialist in rehabilitation services for persons with SCI. In the next step, one of the authors rated the quality of translation (0 - 100) and reproduced the final revision of translations as a final version besides two experts. The final version of the scale translated from Persian into English and has been sent to the main developer in order to maintain his approval and further studies took place after his agreement on the translated instrument (Appendix 1 in Supplementary File).
3.2. Participants
The sample size of the population selected according to the rule of thumb principle, which indicates five samples for every item in the questionnaire, and it should not be less than 100 samples, according to Gorsuch (15). The final sample consisted of 50 healthy individuals and 150 persons with SCI. The healthy individuals selected from available individuals aged from 20 to 65, with different genders and social and academic levels. In this study, the sampling method was simple, non-random and available. One of the authors administered the completion of the scale by participants, in order to ensure the comprehensiveness of items. Inclusion criteria included having at least 18 years old, the ability of reading and writing in Persian, the cognitive ability to handle questions, and their consent for participating in the study.
The informed consent was obtained from the participants via clarifying the purpose, risks, benefits and the process of the study. The authors confirm they have appropriately handled confidentiality and data security.
3.3. Instruments
USER-P consists of 31 items divided into three sub-scales (Frequency, Restriction, Satisfaction). The frequency sub-scale consists of two parts; Part A determines the number of hours engaged in four general activities during a typical week of individual’s life, and Part B asks the number of times engaged in eight social and leisure activities, such as going out, visiting family and friends, and indoor hobbies in the past four weeks.
In Part A of the frequency sub-scale, the scores range indicate 0 (not at all) to 5 (36 hours or more); and in Part B, from 0 (not at all) to 5 (19 times or more).
The restriction sub-scale consists of 10 items which indicate the experienced participation restrictions as a result of the injury or disability, in the social, vocational and leisure activities. Each item of the restriction sub-scale ranges from 0 (not possible at all) to 3 (independent without difficulty). There’s a “not applicable” answer for each item in case of being irrelative to the person’s situation or if the experienced restriction does not relate to the person’s injury or disability. The satisfaction sub-scale consists of 9 items which indicate the satisfaction with social relationships, vocational activities and leisure. Items of the satisfaction sub-scale range from 0 (very dissatisfied) to 4 (very satisfied). The total score of each sub-scale calculates by the items that were applicable to the participant’s context and situation; the sum score for each sub-scale should be converted to a 0 - 100 scale. Higher scores demonstrate higher levels of participation. There is no total score for the USER-Participation (11).
Data was collected from April to September 2019, and analysis study took place afterwards until December 2019.
3.4. Validity and Reliability Assessment
The process of validity and reliability took out through the following steps: Face validity (cognitive interview), content validity (qualitative), Discriminant validity, and then for reliability, we used internal consistency (Cronbach’s alpha) and test-retest by intraclass correlation coefficients (ICC).
Face validity conducted by partaking 10 SCI persons with elementary education. Initial assessment guided the authors to re-evaluate the items based on the understanding of the target population. In the next step, qualitative content validity method conducted by a panel of 10 experts including English-fluent rehabilitation and healthcare professors, experienced with the adaptation of evaluation instruments. The panel assessed the content validity of the scale by reviewing the items in terms of relevancy of each item to the domains of participation based on content validity procedure. Discriminant validity performed between healthy and SCI individuals, whilst two-independent samples t-test used. In order to verify the reliability and validity of the native instrument and applicability to the target population, assessment of the psychometric properties took place. Cronbach’s alpha used for assessing internal consistency.Paired T-tests were used for test-retest analysis, and ICC implied for assessing the stability of scores over time. To assess reproducibility, 30 individuals from the sample retook the scale after two weeks from the initial administration. Test-retest reliability evaluated by using ICC between the initial and second assessment. Values less than 0.5, between 0.5 and 0.75, between 0.75 and 0.9, and greater than 0.90 were indicative of poor, moderate, good, and excellent reliability, respectively (16). The socio-demographic information gathered using an additional one-page form. Statistical analysis conducted by Statistical Package for the Social Sciences (SPSS) Version 22.
4. Results
4.1. Participant Characteristics
Most of the population were male (59.7%). One-third of the participants did not continue higher education after diploma (33.3%), and 18.7% of them did not graduate from high school. In terms of insurance, about half of the participants were covered by basic health insurance, and more than half (62.7%) of them were provided by the services of social welfare organization. Full demonstration of participants’ characteristics has provided in Table 1.
Characteristics of Total Participants in the Study (150 Persons)
| Variables | Values a |
|---|---|
| Gender | |
| Male | 58 (38.7) |
| Female | 92 (61.3) |
| Education | |
| High school or below | 28 (18.7) |
| Diploma | 50 (33.3) |
| Associate degree | 24 (16) |
| Bachelors | 40 (26.7) |
| Masters | 8 (5.3) |
| Marital status | |
| Single | 39 (26) |
| Married | 89 (59.3) |
| Divorced | 12 (8) |
| Widowed | 10 (6.7) |
| Occupation | |
| Employed | 66 (44) |
| Unemployed | 84 (56) |
| Injury type | |
| Diplegia | 58 (38.7) |
| Paraplegia | 54 (36) |
| Quadrplegia | 12 (8) |
| Triplegia | 11 (7.3) |
| Paraparesia | 15 (10) |
4.2. Translation and Cultural Adaptation
The cultural adaptation included forward translation, evaluation of the translation, and backward translation, consecutively.In the evaluation process, the word “partner” in the items that asked the participants’ relationship with his/her partner, were culturally odd for the ordinary individual, as the courtship only applies in the form of legal marriage in Iranian society; therefore, the word “partner” replaced with the word “spouse”, and then translated into the Persian language to be more understandable for the Iranian participant.
4.3. Evaluation of Psychometric Characteristics
Psychometric characteristics include face and content validity; discriminant validity, reliability that has addressed in separate parts.
4.3.1. Face Validity
In terms of face validity, initial participators which consisted of ten SCI patients with elementary education assigned the fluency and relevancy of the items. The authors then re-evaluated the initial translated version and sent the revised version to the expert panel in order to assess content validity.
4.3.2. Content Validity
In order to enrich the cultural adaptation process, several modifications were made to enhance the fluency of scale, especially for those participants who have had elementary education. For instance, several descriptions of the items were modified with respect to the context of the Iranian participant. Outdoor activities such as going to the beach, concert and/or attending church were replaced by more common activities in Iranian culture such as going to the mosque and attending public social events (Table 2).
Expert Panel Suggestions on Qualitative Content Validity
| Number | Item | Panel Recommendation | Final Correction |
|---|---|---|---|
| 1 | Sport (such as: Tennis, Bicycling, and gym) | Examples of this item should adapt for a typical spinal cord injury person | Main examples replaced by Sitting volleyball and cycling with wheelchair |
| 2 | Going Out (e.g. Coffee shop, concert, theatre) | It is recommended to replace the examples of this item with more identical activities in Iranian culture | Mentioned activities have been replaced by cultural events and art performances. |
| 3 | Your relationship with your partner (such as: communication, sexuality) | “Communication” needs to have a more common equivalent in Persian language | The translation of “communication” has been established as equivalent of “reciprocal relation” in Persian language |
| 4 | Attending events, going to the beach, church or mosque | Social events that are more common in Iranian culture should replace the mentioned activities | Attending private gatherings and public events, going to mosque and other religious spots replaced for mentioned examples of going out |
The expert committee proposed several modifications to adapt the items with native culture.
4.3.3. Discriminant Validity
In terms of discriminant validity, the difference between groups that are certainly distinct in terms of the questionnaire’s construct get compared (17). Subsequently, scores of 50 normal individuals, got compared with 150 SCI persons who had completed P-USER-P (18). According to the P-value which is smaller than 0.001, all three sub-scales had meaningful discriminant coefficients. (Table 3).
Compare Sub-scales Scores Between Normal-Range and Persons with Spinal Cord Injury (SCI)
| Sub-scale | N | Mean ± SD | t | P-Value |
|---|---|---|---|---|
| Frequency | -6.051 | < 0.001 | ||
| Normal | 150 | 27.35 ± 4.216 | ||
| SCI | 50 | 32.98 ± 6.116 | ||
| Restriction | -11.717 | < 0.001 | ||
| Normal | 150 | 30.65 ± 4.685 | ||
| SCI | 50 | 40.14 ± 5.721 | ||
| Satisfaction | -12.443 | < 0.001 | ||
| Normal | 150 | 27.29 ± 4.426 | ||
| SCI | 50 | 36.28 ± 4.426 |
4.3.4. Reliability
Internal consistency of the P-USER-P was assessed using Cronbach’s alpha. The ICC and cronbach’s alpha values more than 0.7 indicated moderate and good internal consistency in different sub-scales (19). Table 4 demonstrates the results of internal consistency for each sub-scale.
Internal Consistency of P-USER-P Sub-scales
| Scale | Number of Items | Cronbach’s Alpha |
|---|---|---|
| Frequency | 11 | 0.615 |
| Restrictions | 11 | 0.715 |
| Satisfaction | 9 | 0.695 |
The reproducibility of the P-USER-P also verified by 30 SCI persons after two weeks from the initial assessment. In this study, coefficients of internal consistency and test-retest reliability showed generally moderate to very good results. Results of the T-test (Table 5) showed good ICC score in P-USER-P frequency sub-scale (0.755), and very good ICC scores in restrictions (0.849) and satisfaction (0.846) sub-scales between the scores of test and retest.
Test-Retest Reliability Scores of P-USER-P Sub-scales (N = 30)
| Sub-scales | Test, Mean ± SD | Retest, Mean ± SD | Absolute ICC | Upper Bound | Lower Bound | P-Value |
|---|---|---|---|---|---|---|
| Frequency | 27.35 ± 4.21 | 27.23 ± 3.07 | 0.606 | 0.320 | 0.791 | 0.001 |
| Restriction | 30.66 ± 4.69 | 28.2 ± 4.49 | 0.737 | 0.517 | 0.866 | 0.001 |
| Satisfaction | 27.29 ± 4.42 | 28.5 ± 4.02 | 0.734 | 0.512 | 0.864 | 0.001 |
5. Discussion
This study aimed at assessing the cross-cultural adaptation of the USER-P to Persian to develop the P-USER-P and its reliability and validity. While there are few scales that aim to evaluate both subjective and objective participation of rehabilitation patients, P-USER-P counts as a valid reliable instrument that has been translated and validated in at least one Asian language before Persian and has been proven to be congruent with Eastern cultures (20).The USER-P had been also translated into the Korean language and tested in terms of psychometric properties with a sample of stroke patients by Lee et al. (20). They found three subscales of K-USER-P to have Cronbach's alpha between 0.66 and 0.69. Also, in terms of test-retest reliability, they found that the satisfaction scale had 0.71 ICC, and frequency and restriction subscale had 0.63 and 0.45, respectively. These results are compatible with the findings of our study, and also, higher ICC scores indicate that P-USER-P is a more reliable tool than K-USER-P. The results of the reliability study were compatible with the psychometric study of original USER-P which was conducted by Mader et al. on a population of rehabilitation outpatients in The Netherlands (8). Their research showed Cronbach’s alpha of 0.65 for the Frequency, as well as 0.90 for both satisfaction and restrictions subscales. The only advantage of Mader et al.’s research was their higher value of alpha for Restriction and Satisfaction scales which were about 0.2 higher (8). Another psychometric study of original USER-P by Post et al. found that internal consistency of the Restrictions and Satisfaction scales was very good, alphas were high and all item–rest correlations well above the criterion (11). Similarly, we found the similar values for Restriction and Satisfaction subscales.
Face and content validity performed by applying minor changes in translation to increase the comprehensiveness of items within the cultural context of Iran.
In terms of discriminant validity, our research showed that all subscales of P-USER-P distinguish between normal-range and SCI adults. Van der Zee et al. (21) in their discriminant validity examination concluded that USER-P demonstrated significant differences in levels of participation between persons with different health conditions and different levels of functional limitations, overall, the Restrictions score was sensitive to variations in their research (5). In another relevant study, participation measure for post-acute care (PM-PAC) questionnaire, showed generally varied subscale scores on the concept of condition severity and the ability for mobility (22). Overall, the initial goals of the research obtained.
The strengths of this study are its unique and comprehensive thoroughness in following the cultural and linguistic adaptation, and the findings of the authors and translators at reaching a consensus version. Furthermore, the final version was dedicated to a Persian-speaking SCI population. The limitations of this study include the lack of a Persian instrument similar to USER-P for assessing the concurrent validity, therefore this investigation is highly recommended in the future researches. Also, the instalment of a proper online communication pathway for accessing the persons with SCI would enhance the productivity of the future researches.
5.1. Conclusions
Community administrators and rehabilitation specialists should adapt their health and wellbeing plans based on indicators that enable the individuals to attempt in beneficial activities that can support a person’s livelihood and the dynamic economy. Results of this study are important because the P-USER-P can be used by occupational therapists and other rehabilitation specialists to measure the level of participation in persons with SCI in Iran. The findings of this study approves P-USER-P as a valid and reliable instrument for evaluation of Rehabilitation-Participation in an Iranian SCI population.
Acknowledgements
References
-
1.
World Health Organization. International classification of functioning, disability and health: children and youth version: ICF-CY. WHO; 2007. Available from: https://apps.who.int/iris/handle/10665/43737.
-
2.
Piskur B, Daniels R, Jongmans MJ, Ketelaar M, Smeets RJ, Norton M, et al. Participation and social participation: are they distinct concepts? Clin Rehabil. 2014;28(3):211-20. [PubMed ID: 23988324]. https://doi.org/10.1177/0269215513499029.
-
3.
Wolf AC, Tate RL, Lannin NA, Middleton J, Lane-Brown A, Cameron ID. The World Health Organization disability assessment scale, WHODAS II: reliability and validity in the measurement of activity and participation in a spinal cord injury population. J Rehabil Med. 2012;44(9):747-55. [PubMed ID: 22854805]. https://doi.org/10.2340/16501977-1016.
-
4.
World Health Organization. World report on disability 2011. WHO; 2011. Available from: https://www.who.int/teams/noncommunicable-diseases/sensory-functions-disability-and-rehabilitation/world-report-on-disability.
-
5.
Javanmard A, Abdi K, Ebadi A, Hosseinzadeh S. Participation instruments in persons with spinal cord injury: A narrative review. Med J Islam Repub Iran. 2020;34:66. [PubMed ID: 32974232]. [PubMed Central ID: PMC7500424]. https://doi.org/10.34171/mjiri.34.66.
-
6.
Cardol M, Brandsma JW, de Groot IJ, van den Bos GA, de Haan RJ, de Jong BA. Handicap questionnaires: what do they assess? Disabil Rehabil. 1999;21(3):97-105. [PubMed ID: 10206348]. https://doi.org/10.1080/096382899297819.
-
7.
Van De Velde D, Bracke P, Van Hove G, Josephsson S, Viaene A, De Boever E, et al. Measuring participation when combining subjective and objective variables: the development of the Ghent Participation Scale (GPS). Eur J Phys Rehabil Med. 2016;52(4):527-40.
-
8.
Mader L, Post MW, Ballert CS, Michel G, Stucki G, Brinkhof MW. Metric properties of the Utrecht Scale for Evaluation of Rehabilitation-Participation (USER-Participation) in persons with spinal cord injury living in Switzerland. J Rehabil Med. 2016;48(2):165-74. [PubMed ID: 26926920]. https://doi.org/10.2340/16501977-2010.
-
9.
Noonan VK, Kopec JA, Noreau L, Singer J, Masse LC, Zhang H, et al. Comparing the validity of five participation instruments in persons with spinal conditions. J Rehabil Med. 2010;42(8):724-34. [PubMed ID: 20809054]. https://doi.org/10.2340/16501977-0584.
-
10.
Noonan VK, Miller WC, Noreau L, Scire Research Team. A review of instruments assessing participation in persons with spinal cord injury. Spinal Cord. 2009;47(6):435-46. [PubMed ID: 19238163]. [PubMed Central ID: PMC3535488]. https://doi.org/10.1038/sc.2008.171.
-
11.
Post MW, van der Zee CH, Hennink J, Schafrat CG, Visser-Meily JM, van Berlekom SB. Validity of the utrecht scale for evaluation of rehabilitation-participation. Disabil Rehabil. 2012;34(6):478-85. [PubMed ID: 21978031]. https://doi.org/10.3109/09638288.2011.608148.
-
12.
Chiodo AE, Scelza WM, Kirshblum SC, Wuermser LA, Ho CH, Priebe MM. Spinal cord injury medicine. 5. Long-term medical issues and health maintenance. Arch Phys Med Rehabil. 2007;88(3 Suppl 1):S76-83. [PubMed ID: 17321853]. https://doi.org/10.1016/j.apmr.2006.12.015.
-
13.
Noreau L, Fougeyrollas P, Post M, Asano M. Participation after spinal cord injury: the evolution of conceptualization and measurement. J Neurol Phys Ther. 2005;29(3):147-56. [PubMed ID: 16398947]. https://doi.org/10.1097/01.npt.0000282247.15911.dc.
-
14.
Farajzadeh A, Akbarfahimi M, Maroufizadeh S, Rostami HR, Kohan AH. Psychometric properties of Persian version of the Caregiver Burden Scale in Iranian caregivers of patients with spinal cord injury. Disabil Rehabil. 2018;40(3):367-72. [PubMed ID: 28637138]. https://doi.org/10.1080/09638288.2016.1258738.
-
15.
Gorsuch RL. Factor analysis: Classic edition. 2nd ed. New York: Routledge; 2014.
-
16.
Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155-63. [PubMed ID: 27330520]. [PubMed Central ID: PMC4913118]. https://doi.org/10.1016/j.jcm.2016.02.012.
-
17.
Lucas RE, Diener E, Suh E. Discriminant validity of well-being measures. J Pers Soc Psychol. 1996;71(3):616-28. [PubMed ID: 8831165]. https://doi.org/10.1037//0022-3514.71.3.616.
-
18.
Hattie J, Cooksey RW. Procedures for assessing the validities of tests using the "known-groups" method. Appl Psychol Meas. 2016;8(3):295-305. https://doi.org/10.1177/014662168400800306.
-
19.
Polit DF, Yang F. Measurement and the measurement of change: a primer for the health professions. Philadelphia, PA: Wolters Kluwer; 2016.
-
20.
Lee JH, Park JH, Kim YJ, Lee SH, Post MWM, Park HY. Validity and reliability of the Korean version of the Utrecht scale for evaluation of rehabilitation-participation. Occup Ther Int. 2017;2017:9452051. [PubMed ID: 29097981]. [PubMed Central ID: PMC5612671]. https://doi.org/10.1155/2017/9452051.
-
21.
van der Zee CH, Post MW, Brinkhof MW, Wagenaar RC. Comparison of the Utrecht scale for evaluation of rehabilitation-participation with the ICF measure of participation and activities screener and the WHO disability assessment Schedule II in persons with spinal cord injury. Arch Phys Med Rehabil. 2014;95(1):87-93. [PubMed ID: 24008052]. https://doi.org/10.1016/j.apmr.2013.08.236.
-
22.
Gandek B, Sinclair SJ, Jette AM, Ware JJ. Development and initial psychometric evaluation of the participation measure for post-acute care (PM-PAC). Am J Phys Med Rehabil. 2007;86(1):57-71. [PubMed ID: 17033591]. https://doi.org/10.1097/01.phm.0000233200.43822.21.