Translation, Validity, and Reliability of Singing Voice Handicap Index in Persian Traditional Singers: A New Revised Version

1. Background

It is acknowledged that voice disorders can cause poor quality of life for professional voice users such as singers. Singers constitute 11.5% of clients of some voice clinics while representing only 0.02% of the general population (1). As a working population that relies on voice in their profession, singers have higher vocal demands and are more at risk of developing vocal disorders than the general population (2).

According to the World Health Organization (WHO), the handicap resulting from any disability may exert occupational, social, or economic effects (3). Accordingly, a valid and reliable vocal disorder scale for singers needs to evaluate all aspects of handicap and be sensitive to all levels of function. A valid and reliable vocal disorder scale for singers could guide voice service providers to capture different aspects of singers' voice problems, in particular, to address specific problems in the treatment process, and evaluate the effect of management on a voice index over the therapeutic process.

Persian traditional singing is the original, “popular” singing style in Iran (4), where “popular” is understood as “of the people,” perhaps best translated as “folk.” "Tahrir/Tahreer" is musical ornamentation similar to vibrato, produced in this singing style (4). Production of diverse Tahrir/Tahreer styles is a delicate and crucial vocal skill that may be affected by vocal impairment (5). Moreover, since Tahrir/Tahreer production may create uncomfortable laryngeal muscle tension, traditional singers may be more exposed to vocal impairment than many others (6). Along similar lines, traditional Persian singing codifies songs according to melodic figures called "Radif." These figures are transmitted orally across generations. Iran's traditional music is based on Radif. The acquisition of these figures typically requires years of practice (see, for example, here, making the learning process different from the process involved in most other singing styles (6). This learning process may place traditional Persian singers at greater risk of voice disorders than others (6).

The singing voice handicap index (SVHI) is a commonly used measure for clinical purposes, developed and validated by Cohen et al. (7). The Persian translation of the SVHI (P-SVHI) assesses symptoms commonly reported to laryngologists and speech-language pathologists by singers.

The SVHI has been translated into various languages (3, 8-16). The validation of the P-SVHI was previously investigated by Ghaemi et al. (15) in all singing styles. However, their study did not provide evidence regarding the validity and reliability of the SVHI for traditional Persian singers specifically, who comprise a vast majority of singers in Iran. Moreover, the study did not assess the construct validity to explore relevant dimensions of the P-SVHI.

2. Objectives

Although Ghaemi et al. study represented an initial step in assessing singing disruptions in Persian singers (15), considering traditional singers' particular vocal demands, the current study aimed to investigate the psychometric properties of SVHI in traditional Persian singers.

3. Methods

3.1. Study Design and Participants

A methodological study was conducted from September 2019 to February 2020 after obtaining approval from the Research Ethics Committee. A convenience sample of 285 traditional Persian singers (222 males and 63 females) from 10 music and singing centers in Tehran was recruited over six months.

Inclusion criteria for all participants were as follows: having at least four years of singing performance experience, the age range of 16 to 85 years, and no history of hearing loss or smoking. Participants with the following criteria were excluded from the study: having any forms of rhinitis or common cold at the time of participation.

3.2. Procedure

3.2.1. Translation

After obtaining permission from the scale developer (Cohen), cross-cultural adaptation using a “forward-backward” procedure was applied to translate the original version of SVHI into Persian according to Beaton et al. guidelines (17). Two translators, who had an M.Sc. degree in speech-language pathology, synthesized two initial translations and resolved any inconsistencies. The synthesized Persian version of the SVHI was then back-translated into English by two bilingual translators and a native English speaker unfamiliar with the original English version. The two back translations were assessed by our expert committee, including three speech-language pathologists who compared them to the original version, approved by the developer. The final P-SVHI was developed after approval. As the P-SVHI was available, it was unnecessary to evaluate the face validity. However, the understanding of the questionnaire was done during the translation process in the current study.

As the face validity of P-SVHI was previously approved using the impact score by multiplying frequency and importance (score 4 or 5) ratings for each item in the previous P-SVHI, first, we compared our version with the previous Persian version. All items were approved by the previous version after some changes in wording. Face validity evaluation was considered a step of cross-cultural adaptation in ensuring the quality of the P-SVHI. Therefore, in this section, we applied our P-SVHI to assess the items' clarity and unambiguity using a qualitative process by 21 traditional singers who were asked to determine ambiguous items and suggest other sentences in a pilot study if needed. This section resulted in no change in any of the items.

3.2.2. Data Collection

Subjects completed a demographic questionnaire and P-SVHI. To evaluate the P-SVHI test-retest reliability, 35 healthy and 10 dysphonic singers completed it again in a seven-day interval. Two clinicians were randomly interviewed with 96 subjects with 0.87 Kappa agreement, and they rated voice quality separately using the grade, roughness, breathiness, asthenia, and strain (GRBAS) Scale on three tasks of vowel prolongation, counting to 10, and reading the rainbow text. The GRBAS Scale was previously used with Persian speakers by Hosseinifar et al. (18). Subjects were also asked about their vocal health status within the past month on a binary scale (with voice problem, without voice problem).

3.3. Measures

3.3.1. Singing Voice Handicap Index

The SVHI comprises 36 questions rated on a five-point scale from 0 (never) to 4 (always), with Cronbach alpha of 0.97 of the first article. The total score ranges between 0 and 144, with higher scores indicating more voice disability (3, 7). Sobol et al. reported the cutoff of the original SVHI as 20.35 (95% CI: 10.6 - 30.1) (19). In a previous version of the P-SVHI, the internal consistency was confirmed for Iranian singers generally (Cronbach α = 0.83) (15).

3.3.2. Iowa Patient's Voice Index

The Iowa Patient's voice index (IPVI) was developed by Verdolini et al. cited in Karnell et al. with three times (20). The first two items, "IPVI1" and "IPVI2," are rated on a 0 - 6 scale (0 for "no impact" and 6 for "great impact"). The third item, “IPVI effort,” rates relative vocal effort. On this scale, 100 indicates comfortable effort, 200 indicates twice as much effort as comfortable, and so on (20).

3.3.3. Grade, Roughness, Breathiness, Asthenia, and Strain Measurers

The GRBAS is an auditory-perceptual voice assessment scale that addresses (overall) grade (of vocal dysphonia), roughness, breathiness, asthenia, and strain (21, 22). Clinicians rate these subscales from 0 to 3, in which zero indicates normal, and 3 implies severely impaired voice quality (21). The grade score was used to determine the clinicians’ evaluations of subjects' voices in this study. Samples were further partitioned into normal (0 scores), slightly, moderately, and severely dysphonic groups (1, 2, and 3 scores, respectively) based on the GRBAS Scale (23).

3.4. Statistical Analysis

Descriptive statistics were obtained for all variables with SPSS software ver.16 (SPSS Inc., Chicago, IL). We used descriptive analysis to calculate the mean ± SD and linear regression to assess the correlation between quantitative variables. There were no missing data. Univariate distribution of data was used to evaluate normality. Skewness (± 3) and kurtosis (± 8) were applied to evaluate the univariate normal distribution. The independent-sample t-test was used to assess the differences between healthy singers and individuals with a vocal disorder. A one-way analysis of variance (ANOVA) with Tukey post hoc test was used to assess the difference between different levels of P-SVHI based on the GRBAS diagnostic tool. The statistical significance level was P < 0.05. We followed COSMIN recommendations to report this study (24).

3.5. Psychometric Analysis

3.5.1. Validity

We assessed face, construct, discriminant, and convergent validity, as follows.

3.5.1.1. Construct Validity

An exploratory factor analysis was employed using the maximum likelihood method with Promax rotation (oblique). The extraction of each factor was determined based on a factor loading of almost 0.31, which was estimated using the following formula:

CV = 5.152 ÷ √ (n – 2)

Where CV is the number of extractable factors and n is the sample size (25). The Eigenvalue was considered ≥ 1 (26). The sample size should be more than 200 participants for factor analysis, according to MacCallum et al. (27). However, we recruited 285 cases.

Items with communalities less than 0.2 were excluded from the analysis. Bartlett's test of sphericity was performed to evaluate the appropriateness of the study sample. The Kaiser-Meyer-Olkin (KMO) test was also used to assess the data for factor analysis with a cutoff of 0.9 as an excellent result.

3.5.1.2. Scoring

To facilitate the interpretation of the results and compare them to other studies, we used a linear transformation to homogenize the scoring of original SVHI scores with the new P-SVHI using the following formula:

Score in 0 - 100 = ((The obtained raw score-The lowest possible raw score) / (The highest possible score - The lowest possible score)) × 100

The final scoring of this transformation is 0 - 100 (28).

3.5.1.3. Discriminant Validity

We used known-group analysis to investigate the clinical validity of the tool and differences between healthy and dysphonic singers. Then, we assumed that singers with a voice disorder would have higher scores on the P-SVHI than healthy singers. Since the P-SVHI scores showed normal distribution, an independent sample t-test was used.

The GRBAS was used as a diagnostic tool among 96 participants. No one received the three scores of GRBAS. Thus, we compared P-SVHI scores to GRBAS scores. A one-way analysis of variance (ANOVA) assessed statistical differences in P-SVHI scores across the GRBAS levels.

3.5.1.4. Convergent Validity

We expected that if the questionnaire was valid, there would be a correlation between P-SVHI and IPVI. It was assessed randomly among 179 participants. A Spearman correlation was calculated between the total score and subscale scores of P-SVHI and IPVI test items. A moderate correlation coefficient (r = 0.3) between the relevant scales was acceptable (29).

3.5.2. Reliability

3.5.2.1. Test-retest Reliability and Internal Consistency Analysis

The reliability of the P-SVHI was determined using (a) internal consistency with the average inter-item correlation (AIC) and alpha (30) and (b) test-retest analysis using the intra-class correlation coefficient (ICC_{two-way random, absolute agreement}: ICC_2,k) (31) using two-way random model, with interaction for the absolute agreement between scores (ICC_2,k) (32). A cutoff of 0.70 was considered acceptable, a value < 0.8 was considered good, and a value > 0.9 was considered the evidence of excellent internal consistency (25). Item internal consistency was assessed using item and subscale correlation corrected for overlap, and the Spearman rank correlation coefficient was used (33); a value of 0.40 was considered acceptable, and ≥ 0.80 indicated high internal consistency (34). The ICC_2,K values < 0.4 showed poor agreement, between 0.4 and 0.75 fair to good agreement, and values > 0.75 excellent agreement (35).

3.5.3. Floor and Ceiling Effects

The distribution of responses was evaluated by assessing potential floor and ceiling effects. These effects were considered at the lowest (0) and highest (4) scores. A floor or ceiling effect was present if more than 15% of the respondents indicated the lowest or the highest possible score on the P-SVHI (36).

4. Results

4.1. Demographic Data

Of 300 traditional singers, 15 were excluded due to either rhinitis comorbidity or smoking status. Of all 285 participants, we selected 275 traditional singers because 10 persons had outlier data. Therefore, the final analysis was done among 168 healthy subjects and 107 subjects who experienced a voice disorder in the last month. The mean ± SD of age was 35.43 ± 10.7, and the singing experience was 12.06 ± 13.27 years. The majority of the subjects (77.9%) were males. All characteristics are shown in Table 1.

4.2. Validity

4.2.1. Construct Validity

All Skewness values were between ± 3, and kurtosis values were between ± 7. Bartlett's test of sphericity (χ² = 6410.07, P-value < 0.001) and Kaiser-Meyer-Olkin test (KMO = 0.91) demonstrated the appropriateness of sampling. The exploratory factor analysis extracted four factors with eigenvalues > 1.0 (Table 2). None of the items had communalities less than 0.2. Items 3, 7, 8, 13, 15, 19, 20, and 27 were removed from analysis according to the following method: first, we removed items that loaded on two factors from EFA one by one if their factor loadings had less than 0.2 differences and re-run factor analysis to identify the next item should be removed. Therefore, because the cross-loadings persisted, other items became candidates for deletion. Therefore, these items were excluded based on the EFA results (37). The introducing factors of the final 29-item P-SVHI (Table 2) followed factor 1 including 10 items (5, 21, 10, 6, 23, 31, 14, 26, 17, and 24), factor 2 including seven items (35, 34, 36, 32, 28, and 33), factor 3 with seven items (2, 4, 18, 9, 1, and 16), and factor 4 including three items (29, 25, and 30). In the final structure, item 25 was cross-loaded on factors 1 and 2, but the difference between factor loadings was more than 0.2. Therefore, item 25 belonged to factor 4. A similar pattern was also seen in factor loadings of item 31. Therefore, it was considered an item of factor 1.

4.2.2. Scoring

We used the linear transformation to score the new P-SVHI. Since 7 items were eliminated after factor analysis, the range of the total score of P-SVHI decreased from 0 - 144 in the original 36-item version to 0 - 100 in the 29-item P-SVHI. Moreover, the mean ± SD score was 16.00 ± 14.08 for healthy singers and 40.00 ± 17.05 for singers with a voice disorder. After performing the linear transformation, the scoring of the 29-item P-SVHI was transferred from 0 - 144 to 0 - 100. The mean ± SD total score was 26.26 ± 19.87. Therefore, in a 0 - 100 scoring, healthy subjects obtained a lower mean score than the dysphonic participants (16.9 vs. 40.9).

4.2.3. Discriminant Validity

The means ± SD of P-SVHI scores were statistically different between healthy (16.90 ± 14.9) and dysphonic (40.9 ± 17.60) singers (P-value < 0.001) (Figure 1).

Difference between SVHI scores of healthy and dysphonic singers

Figure 1.

Difference between SVHI scores of healthy and dysphonic singers

In a further approach, we randomly used the GRBAS Scale among 96 participants to discriminate between the total P-SVHI and GRBAS scores. A significant difference between the P-SVHI total and three levels of GRBAS scores was observed using one-way ANOVA (F = 92.2, P < 0.000). The P-SVHI total score was 18.3 ± 10.1 in singers with normal voice (GRBAS = 0), 39.2 ± 7.2 in singers with slight voice problems (GRBAS = 1), and 58.8 ± 7.3 in singers with moderate voice problems (GRBAS = 2). The Tukey post hoc test revealed that these significant differences were related to all levels of the GRBAS Scale (P-values 0 < 0.001) (Figure 2).

Average P-SVHI score according to different levels of GRBAS score

Figure 2.

Average P-SVHI score according to different levels of GRBAS score

4.2.4. Convergent Validity

The correlations between the P-SVHI total score and IPVI1, IPVI2, and "IPVI effort" scores were r = 0.52, 0.48, and 0.33, respectively. The correlation between IPVI items and all P-SVHI subscales ranged from r = 0.30 to 0.62, as shown in Table 3.

4.3. Reliability

4.3.1. Test-retest Reliability and Internal Consistency Analysis

The average inter-item correlation (AIC) was 0.4, and Cronbach's alpha was 0.95 for the total scale. Test-retest analysis using the intra-class correlation coefficient (ICC_2,k) ranged from low (0.69) to high (0.95) (Table 4).

The ICC_2,K value for the total score was 0.97 (P < 0.001). The lowest ICC_2,K was 0.69, and the greatest was 0.95. Corrected total item correlation was calculated for each item, ranging from 0.69 to 0.95. Therefore, all remaining 29 items met the acceptance criteria (ICC_2,K > 0.4) and were retained in the new P-SVHI.

4.4. Floor and Ceiling Effects

The total floor and ceiling factor effects were 7% and 0%, respectively.

5. Discussion

Because of particular voice demands, specific tools should be used to evaluate potential occupational handicaps in singers (7). In the current study, the psychometric properties of the P-SVHI, with a focus on factor analysis, were investigated in traditional Persian singers. We found that the 29-item P-SVHI is valid and reliable for traditional Persian singers.

Concerning the demographic features of subjects, there was a significant difference between the number of male and female subjects. One reason may be that in Iran, female singers are only allowed to perform in public in choir groups, and they are not allowed to perform as solo singers unless for female audiences.

Investigating the construct validity of the original SVHI in English, Cohen et al. (2007 and 2009) performed Principal Component Analysis (PCA) by the quartimax and varimax methods to assess the potential subscales of the SVHI and then develop the short form of the SVHI Scale (7, 38). It yielded a different number of constructs. One of the significant issues of Cohen et al. studies is the rotation method applied in EPA (7, 38). By considering the essential association between given factors and an inter-correlation between factors extracted, oblique rotation is preferred over varimax and quatrimax. Accordingly, we used the Promax method as one of the most potent ways to rotate X and Y axes with a non-orthogonal angle to get a proper structure of the items. Another issue that needs to be addressed in Cohen et al.'s study is that they used principal component analysis. Once the concept underlying a questionnaire such as SVHI is extracted into various items, the maximum likelihood method seems to be the proper method of performing explanatory factor analysis (EFA). At the same time, PCA is used when some factors define the central concept of an index (34). However, these differences may be related to either the method of EFA running or applied rotation. In addition, based on the formula presented in the construct validity section, we used the ≥ 0.3 value as a cutoff point to extract factors through the EFA, while Cohen et al. study considered a threshold of 0.4 for factor loading instead of 0.30. Thus, extracting only one factor by Cohen et al. (7, 38) compared to four factors extracted by the current study might be related to the cutoff they chose. Moreover, Cohen et al. omitted 26 items of the original 36 items, including items 2, 3, 4, 5, 8, 9, 11, 12, 13, 14, 16, 19, 20 - 28, 30, and 33 - 36 considering consensus comments, items being loaded on two factors, presence of low item-total correlation, presence of low factor loading, and floor effects lower than 0.5. In the current study, items 3, 7, 8, 13, 15, 19, and 27 were omitted after Promax oblique rotation, five of which were also deleted by Cohen et al. in their next article (38). An oblique rotation allows a degree of correlation between the factors to improve the inter-item correlations within the factors. Cross-loading of seven items on two factors reduced the items from 36 to 29 items. Therefore, we obtained a new version of P-SVHI with 29 items that seemed appropriate to traditional singers.

We converted the total score of all these studies into a 0 - 100 score so the results would be comparable to ours. The mean score for healthy singers in the current study was 16.90 on the P-SVHI score, range 0 - 100, which is similar to the results of healthy singers in Sobol et al., in which 729 healthy singers pooled from different languages had an average score of 14.10 on a 0 - 100 scale (19). The mean P-SVHI score, range 0 - 100, in healthy singers in the current study (16.9) also is similar to the results previously reported by Ghaemi et al. (15). The mean ± SD scores of the SVHI were reported in Spanish (voice disorder patients vs. healthy group: 61.46 ± 30.50 vs. 28.43 ± 18.58) (13), Korean (dysphonic vocalist group vs. normal vocalist group: 70.63 ± 33.90 vs. 19.84 ± 12.84) (16), Kannada (voice problem group vs. normal group: 61.05 ± 20.57 vs. 19.81 ± 10.74) (9), Persian (voice disorder patients vs. healthy group: 68.75 ± 10.02 vs. 18.0 ± 2.65) (15), and Turkish (pathology group vs. healthy group: 53.6 ± 28.9 vs. 21.8 ± 18.5) (11). Also, the P-SVHI score (0 - 100) for dysphonic subjects in our study was 40.9, which is more than the results for an Italian version in which dysphonic signers scored 31.68 (converted to 0-100), suggesting that voice disorder in our traditional singers is more than that in other style singers. In the Italian version, the mean ± SD scores of SVHI in the normal group of singers and vocal fold abnormality group were 29.26 ± 25.72 and 45.62 ± 27.95, respectively.

There was a significant and moderate correlation between P-SVHI and IPVI scores, implying that singers with higher IPVI scores had higher P-SVHI scores. Similar results were shown in the Korean version of SVHI, in which Lee et al. reported that the correlation coefficient between the SVHI score and IPVI’s first item was 0.52 for normal vocalists and 0.78 for dysphonic vocalists (16). As the IPVI reflects the singers’ perception of the quality of their singing voice, we logically expected to observe the relationship between the voice quality and all the subscales of the SVHI.

Robust test-retest reliability was also shown for the 29-item P-SVHI with a high value of ICC_2,k (0.63 - 0.95). Internal consistency was confirmed through a Cronbach alpha coefficient and average inter-item correlation, similar to the previous version in the Ghaemi et al. study (Cronbach α = 0.83) (15).

Our study found no floor or ceiling effects for the P-SVHI total score. If floor and ceiling effects are present for a questionnaire, they may influence the tool’s responsiveness; as a result, it cannot identify singers' improvement and deterioration beyond the extreme ends of the scales. Therefore, the lack of floor or ceiling effects in our study reflects the reliability of the P-SVHI.

5.1. Limitations

One of the limitations of the current study was that only an auditory perceptual examination was performed to distinguish between healthy and dysphonic singers. Therefore, to better distinguish between healthy and disordered people, we suggest a comprehensive laryngeal examination performed by a laryngologist. Besides, another limitation of the current study is that only traditional Persian singers were included. Thus, to better understand vocal handicaps in Persian singing styles, investigating the psychometric features of the P-SVHI in different singing styles is proposed. Moreover, to better understand the effect of various vocal pathologies on vocal handicap, the current study researchers suggest that participants be categorized based on different voice disabilities.

5.2. Conclusions

This study showed that the revised 29-item P-SVHI is a valid, reliable, and sensitive tool for measuring voice-induced handicaps for traditional Persian singers with four subscales. Also, factor analysis showed that this questionnaire could capture different aspects of vocal handicaps with acceptable divergent and convergent validity. This P-SVHI version is an excellent questionnaire to discriminate between healthy and dysphonic traditional singers. This scale also showed high internal consistency and good test-retest reliability.