Abstract
Background:
The strengths and difficulties questionnaire (SDQ) is one of the most common screening tools to detect children and adolescents who are at risk of mental health problems or have psychiatric disorders. The standard mode of SDQ completion is on paper; however, some studies used phone administration and argued that this is a feasible and valid method that can substitute in-person administration. However, few studies have investigated the validity and reliability of phone administration of the SDQ.Objectives:
The current study aimed to compare the telephone versus paper-and-pencil administration of the SDQ among Iranian child and adolescent outpatients.Methods:
Sixty six parents with children aged 3 to 15 years completed the SDQ questionnaire using paper-and-pencil and telephone interview techniques. The study was performed in 2016, and participants were from the city of Tehran (capital of Iran). Participants were first divided into two groups. Then, one of them first completed the questionnaire using the face-to-face interviews, and 3 to 4 days later, they again completed the questionnaire using the telephone interview. And the vice versa occurred for the other group. The Pearson correlation analysis was used to calculate the correlation between the two methods in each group separately. Intra-class correlation (ICC) analysis was used to investigate the association between the two administration methods.Results:
A significant positive correlation was found between the two types of administration in both groups (P < 0.05). Intra-class correlation coefficients indicated a good correlation between the scores obtained from each method of administration (all P values were < 0.001).Conclusions:
Telephone administration of SDQ is a reliable method for collecting data when studying emotional and behavioral symptoms in children and adolescents attending outpatient psychiatric centers.Keywords
Child and Adolescent Psychometric Properties Strengths and Difficulties Questionnaire Telephone Administration
1. Background
Emotional and behavioral disorders are common among children and adolescents, with a global prevalence of 13.4% (CI 95%: 11.3 - 15.9) (1). It should be noted that many of these problems remain undetected, and these individuals never receive any intervention (1, 2).
Structured psychiatric interviews, employing clinical judgment by psychiatrists or psychologists, are the gold standard for diagnosing mental health problems (3, 4). Since clinical interviews are time-consuming and only trained, clinicians can administer this technique, a short and easy-to-administer questionnaire is valuable for detecting those at increased risk of psychological problems (5).
The strengths and difficulties questionnaire (SDQ) (6, 7) is a widely used screening tool to detect children and adolescents at risk of mental health problems (8) or those with psychiatric disorders (2). The SDQ can discriminate between healthy children and those with psychiatric disorders (6, 7). It has a single form for teachers and parents of children aged 4 to 17 years, while it also contains a self-report version for those aged 11 - 17 years. The questionnaire consists of 25 items that are distributed equally across five subscales as follow: Conduct problems, hyperactivity-inattention, emotional symptoms, peer problems, and prosocial behavior. The SDQ is translated into several languages, and its normative data have been studied in many countries and cultures. Various studies performed in different cultures and settings have reported appropriate reliability and validity for the SDQ (9-14).
Several questionnaires or inventories-based psychological or psychiatric tools can be administered in paper-and-pencil self-administration mode. However, screening projects and follow-up studies on individuals receiving psychiatric or psychological interventions confirmed the usefulness of other modes of questionnaire administration such as mail (15), telephone (16), or web-based (17) methodologies (18-20). The advantage of using telephone calls is being less time consuming and more practical (18-20), which is of crucial importance in longitudinal studies that participants should complete questionnaires in-person. Compared to using telephone calls for completing questionnaires, online administration has some weaknesses (18-20). For example, many people do not have adequate digital literacy and access to such devices in several parts of the world. Besides, in-time consultation is not available to explain the ambiguities of parents.
The standard technique of the SDQ administration is paper-and-pencil; however, some studies have used other techniques such as phone calls (15, 16, 21, 22). For example, Erhart et al. (15) have compared phone and mail administrations of the SDQ to measure emotional and behavioral symptoms of adolescents and reported that both methods are valid. However, the authors reported that the phone call method showed weaker reliability and validity compared to the mail mode. Palmieri and Smith (16) reported that the administration of the SDQ using phone calls resulted in no problem, which indicates the usefulness of the collected data. These studies suggest that telephone administration of the SDQ is feasible and can be considered as an appropriate alternative to in-person administration. However, the evidence for the validity and reliability of administrating SDQ using phone calls is limited.
2. Objectives
The current study aimed to evaluate the retest reliability of telephone administration of the SDQ compared to the usual paper-and-pencil self-administration method in a group of 3 - 15 years old children and adolescents referred to clinical psychiatric settings of Iranian.
3. Methods
3.1. Participants
In this cross-sectional study, 66 parents of children aged 3 to 15 years completed the SDQ using the paper-and-pencil self-administration (face to face) and telephone interview techniques. The mean age of children was 9.42 ± 3.55 years, and 65% of them were male. All respondents were mothers aged 24 to 48 years.
The parents were recruited from two community mental health centers, and two outpatient child and adolescent clinics at psychiatric hospitals in Tehran, the capital city of Iran, in 2016. Each parent participated in two stages of the study: (A) completing the SDQ in-person, and (B) answering the SDQ items while listening to a psychologist reading the items through the phone call.
3.2. Procedure
In each center, a research assistant coordinated the data collection process. All coordinators had an MSc degree in psychology and were experienced in working with families and conducting SDQ. Moreover, to maximize the accuracy and consistency of the data collection process, training and role-playing sessions were held. We assumed that participation in treatment sessions could influence the participants’ answering to the SDQ items. Therefore, half of the individuals were recruited among those attending the centers at their first appointment, and the rest were selected among individuals with scheduled follow-up appointments.
Initially, a list comprising of all 3 - 15 years old children and adolescents was obtained from the clinics. Then, participants were divided into two groups; (A) clients in the waiting list for their first attendance appointment and (B) clients who had follow-ups attendance and were on treatment. The research assistant contacted the parents from the top of each list to obtain consent. The procedure continued until the required sample was completed. The names were ranked based on the appointment time. Therefore, the researches assumed that the names are ranked randomly. For participants of both groups, oral consent was read to the parents, and, if agreeing to participate, the objectives of the study were described. Then, parents were asked to answer the SDQ items. For half of the parents, first, they were asked to complete the questionnaire in-person. In the following, 3 to 4 days later, they were invited to schedule a time to complete the SDQ through a phone call. The vice versa was performed for those in the other group. The parents were asked to answer the SDQ items through the telephone administration technique. The research assistant arranged the procedure of in-person administration in a way to match with the appointment dates in the clinic.
3.3. Measures
3.3.1. Demographic Questionnaire
All participants filled a demographic questionnaire (age and gender of both child and respondent, and the relationship of the respondent with the child).
3.3.2. Strengths and Difficulties Questionnaire (SDQ)
The SDQ contains 25 items that are equally distributed in five scales, as follows: emotional symptoms, conduct problems, hyperactivity/inattention, peer relationships problems, and prosocial behavior. Each item should be answered on a 3-point Likert scale, ranging from zero to 2. Respondents’ total score is equal to the scores of each scale. The total difficulties score is equal to the sum of scores of all scales, except for the prosocial behavior. The parent and teacher version of the SDQ is translated and evaluated in Farsi in Iran, and the normative data are published. The internal consistency and concurrent validity of the SDQ were good, and the cut-offs were comparable to other studies (9). The internal consistency (Cronbach’s-α) of the parent version of SDQ was 0.73. The concurrent validity of the SDQ was assessed by calculating the correlations between SDQ and the corresponding parent/teacher versions of the Child behavior checklist (CBCL). All correlations between SDQ and CBCL scales were highly significant (P < 0.01). The cut point for the abnormality, as defined by percentile of 90, was above 19 in the parental version of the SDQ.
The administration of the self-report version of the SDQ in a community group of Iranian adolescents showed that it’s a useful screening tool to assess the individuals with psychopathology (23). The authors review showed that all published studies that used SDQ in Iran had used the standard in-person method, and we could not find any report of its telephone administration.
3.4. Data Analysis
Descriptive statistics were used to describe demographic characteristics of the participants as well as the SDQ raw scores of participants, clients with the first appointments, and clients with follow-up appointments. The SDQ scores of the two groups were compared using the independent t-test. The Pearson correlation analysis was performed to calculate the correlation between paper-and-pencil self-administration and telephone call administration techniques in each group separately: the total participants, first attenders, and follow up attenders. Finally, an intra-class correlation (ICC) analysis was conducted to investigate the association between the two types of SDQ administration. A P value of < 0.05 was considered statistically significant.
4. Results
The mean age of the children was 9.42 ± 3.55 years, and 65% of them were male. The mean of the total score and score of each scale of the SDQ are presented in Table 1. Some differences were found between the mean scores of the first and follow-ups appointments attendees; however, these differences were not statistically significant, except for the Hyperactivity/Inattention scale in the pen-and-paper administration technique (P = 0.04).
| SDQ Subscales Scores | Total Group (N = 66) | First Appointment Group (N = 33) | Follow-Up Appointment Group (N = 33) | Compare Means |
|---|---|---|---|---|
| T (P Value) | ||||
| Phone administration | ||||
| Emotional symptoms | 4.3 (2.96) | 3.93 (2.73) | 4.66 (3.16) | -0.99 (0.32) |
| Conduct problems | 4.01 (2.18) | 3.81 (2.28) | 4.21 (2.08) | -0.73 (0.46) |
| Hyperactivity/Inattention | 6.24 (2.81) | 5.69 (3.10) | 6.78 (2.4) | -1.59 (0.11) |
| Peer relationships problems | 3.56 (1.93) | 3.39 (1.96) | 3.72 (1.92) | -0.69 (0.48) |
| Prosocial behavior | 6.92 (2.24) | 6.84 (2.57) | 7.0 (1.9) | -.027 (0.78) |
| Total difficulties | 18.12 (6.91) | 16.84 (7.36) | 19.39 (6.29) | -1.5 (0.13) |
| Paper-and-pencil self-administration | ||||
| Emotional symptoms | 4.33 (2.97) | 4.12 (3.04) | 4.54 (2.92) | -0.57 (0.56) |
| Conduct problems | 3.84 (2.12) | 3.54 (1.9) | 4.15 (2.32) | -1.16 (0.25) |
| Hyperactivity/Inattention | 5.9 (2.94) | 5.18 (3.1) | 6.63 (2.63) | -2.05 (0.04b) |
| Peer relationships problems | 3.96 (2.13) | 4.42 (2.29) | 3.51 (1.88) | 1.75 (0.08) |
| Prosocial behavior | 6.78 (2.45) | 6.48 (2.8) | 7.09 (2.05) | -1.0 (0.32) |
| Total difficulties | 18.06 (7.55) | 17.27 (8.33) | 18.84 (6.71) | -0.84 (0.4) |
As shown in Table 2, there was a significant positive correlation between the two types of the SDQ administration in all groups. Besides, according to the intra-class correlation coefficients, the scores calculated using the telephone administration and in-persons interviews had good correlations (see Table 3).
Correlations Between Telephone and Paper-and-Pencil Self-Administration of the SDQ Among the Total, First Appointment, and Follow-Ups Appointment Groups, Separately
| SDQ Subscales Scores | Total Group (N = 66) r (CI) | New Appointment Group (N = 33) r (CI) | Follow-Up Appointment Group (N = 33) r (CI) |
|---|---|---|---|
| Emotional symptoms | 0.78a (0.618, 0.878) | 0.731a (0.431, 0.885) | 0.829a (0.614, 0.929) |
| Conduct problems | 0.620a (0.381, 0.781) | 0.519a (0.105, 0.779) | 0.716a (0.405, 0.878) |
| Hyperactivity/Inattention | 0.811a (0.668, 0.896) | 0.809a (0.575, 0.92) | 0.796a (0.55, 0.915) |
| Peer relationships problems | 0.636a (0.403, 0.791) | 0.696a (0.371, 0.869) | 0.650a(0.296, 0.847) |
| Prosocial behavior | 0.707a (0.506, 0.835) | 0.758a (0.479, 0.897) | 0.616a(0.244, 0.83) |
| Total difficulties | 0.800a, (CI: 0.65, 0.89) | 0.760a (0.483, 0.898) | 0.856a(0.669, 0.941) |
Intra Class Correlations (ICC) Between Telephone and Paper-and-Pencil Self-Administration in the Total Groupa
| SDQ Subscales Scores | Intra-Class Correlation | Intra-Class Correlation | 95% Confidence Interval | |
|---|---|---|---|---|
| Lower Bound | Upper Bound | |||
| Emotional symptoms | Average measures | 0.876 | 0.798 | 0.924 |
| Conduct problems | Average measures | 0.766 | 0.617 | 0.856 |
| Hyperactivity/Inattention | Average measures | 0.895 | 0.829 | 0.936 |
| Peer relationships problems | Average measures | 0.776 | 0.634 | 0.863 |
| Prosocial behavior | Average measures | 0.826 | 0.716 | 0.894 |
| Total difficulties | Average measures | 0.887 | 0.816 | 0.931 |
5. Discussion
The current study aimed to evaluate the comparability of phone administration of the SDQ with the paper-and-pencil techniques. The findings showed no significant difference between these two methods. The agreement between the two types of administration (ICC) for all scales was excellent (with a 95% confident interval for the ICC estimate, values less than 0.5, between 0.5 and 0.75, 0.75 and 0.9, and greater than 0.90 are indicative of poor, moderate, good, and excellent reliability, respectively). Hence, it can be concluded that the telephone administration of SDQ is reliable. Inconsistent with the finding of the current study, Erhart et al. (15) performed a study on parents of children and adolescents aged 8 - 18 years and reported that the SDQ scores obtained in the telephone method were more positively rated compared to the mail method. This inconsistency can be attributed to various sampling techniques used by these studies; we implemented the two modes of administrations on a single sample, while Erhart et al. (15) used different samples for each administration method. They suggested that the individuals’ desire for social acceptance may have forced them to rate more positively worded questions. However, participants in the telephone method were more willing to participate.
We assumed that participation in treatment sessions could influence the accuracy of parents’ answers to the SDQ items. As mentioned previously, some differences were found between the mean scores of the first and follow-ups appointments attendees, but they were not statistically significant (in general, participants of the latter had higher scores), except for the Hyperactivity/Inattention scale in pen-and-paper administration technique (P = 0.04). It can be attributed to the more predominance of externalizing symptoms among parents. However, the significant correlation between the two methods of administrations in both groups, including first and follow-up appointments, did not support this assumption. Therefore, both techniques can be considered as reliable screening tools for emotional and behavioral problems in children and adolescents referred to the psychiatric outpatient clinics, whether for the first or repeat visits. This finding supports the usefulness of the telephone administration of the SDQ to follow up on the youth’s responses to the treatment and emotional/ behavioral problems changing while receiving various interventions.
Some studies have reported the reliability and validity of telephone-based administration of other tools, for example, to screen the mental status of adults (24), psychotic disorders (25), and cognitive ability in children (26). Moreover, web-based assessments used in recent studies have provided reliable data (27). However, telephone administration of questionnaires contains some limitations, including taking place at home that is an informal environment, and the participants cannot make eye contact with items, that both lead to the distraction of respondents. These factors may result in unreliable responses; however, the findings of the present study do not support this argument. Moreover, in line with Palmieri and Smith (16), we did not face any significant problem during the telephone administration of SDQ, and all clients well-cooperated. Based on the results, it can be argued that telephone administration is an easy-to-use technique to implement screenings or measuring outcome(s) in both clinical and research settings.
Generally, the findings support a new era in the field of mental health, i.e., telepsychiatry. In recent years, it has been emphasized that using e-health technologies can improve access to mental health care, both assessment and providing interventions.
5.1. Conclusions
Telephone administration of SDQ is a reliable method for collecting data when studying emotional and behavioral symptoms in children and adolescents attending outpatient psychiatric centers.
5.2. Strengths and Limitations
According to the best knowledge of the authors, this is the first study that compared two different techniques of SDQ administration on children and adolescents. The promising findings of the present study regarding the comparability of the telephone versus paper-and-pencil method of the SDQ administration can help researchers to use this method more frequently, which sounds more feasible than the in-person method. However, since the sample of the current study was not representative, its findings should be generalized with caution.
Acknowledgements
References
-
1.
Mellor D, Cheng W, McCabe M, Ling M, Liu Y, Zhao Z, et al. The use of the SDQ with Chinese adolescents in the clinical context. Psychiatry Res. 2016;30(246):520-6. [PubMed ID: 27821363]. https://doi.org/10.1016/j.psychres.2016.10.034.
-
2.
Goodman R, Ford T, Simmons H, Gatward R, Meltzer H. Using the Strengths and Difficulties Questionnaire (SDQ) to screen for child psychiatric disorders in a community sample. Int Rev Psychiatry. 2003;15(1-2):166-72. [PubMed ID: 12745328]. https://doi.org/10.1080/0954026021000046128.
-
3.
Nordgaard J, Revsbech R, SæBYE DITTE, Parnas J. Assessing the diagnostic validity of a structured psychiatric interview in a first‐admission hospital sample. World Psychiatry. 2012;11(3):181-5. [PubMed ID: 23024678]. [PubMed Central ID: PMC3449355]. https://doi.org/10.1002/j.2051-5545.2012.tb00128.x.
-
4.
Reaven JA, Hepburn SL, Ross RG. Use of the ADOS and ADI-R in children with psychosis: Importance of clinical judgment. Clin Child Psychol Psychiatry. 2008;13(1):81-94. [PubMed ID: 18411867]. [PubMed Central ID: PMC4426195]. https://doi.org/10.1177/1359104507086343.
-
5.
Kovacs S, Sharp C. Criterion validity of the Strengths and Difficulties Questionnaire (SDQ) with inpatient adolescents. Psychiatry Res. 2014;219(3):651-7. [PubMed ID: 25048754]. https://doi.org/10.1016/j.psychres.2014.06.019.
-
6.
Goodman R. The Strengths and Difficulties Questionnaire: a research note. J Child Psychol Psychiatry. 1997;38(5):581-6. [PubMed ID: 9255702]. https://doi.org/10.1111/j.1469-7610.1997.
-
7.
Goodman R. Psychometric properties of the strengths and difficulties questionnaire. J Am Acad Child Adolesc Psychiatry. 2001;40(11):1337-45. [PubMed ID: 11699809]. https://doi.org/10.1097/00004583-200111000-00015.
-
8.
Vostanis P. Strengths and Difficulties Questionnaire: research and clinical applications. Curr Opin Psychiatry. 2006;19(4):367-72. [PubMed ID: 16721165]. https://doi.org/10.1097/01.yco.0000228755.72366.05.
-
9.
Shahrivar Z, Tehrani-Doost M, Pakbaz B, Rezaie A, Ahmadi F. Normative data and psychometric properties of the parent and teacher versions of the strengths and difficulties questionnaire (SDQ) in an Iranian community sample. J Res Med Sci. 2009;14(2):69-77. [PubMed ID: 21772865]. [PubMed Central ID: PMC3129092].
-
10.
Ortuño-Sierra J, Chocarro E, Fonseca-Pedrero E, i Riba SS, Muñiz J. The assessment of emotional and behavioural problems: Internal structure of The Strengths and Difficulties Questionnaire. Int J Clin Health Psychol. 2015;15(3):265-73. [PubMed ID: 30487843]. [PubMed Central ID: PMC6225034]. https://doi.org/10.1016/j.ijchp.2015.05.005.
-
11.
Emam MM, Hilal MMA, Kazem AM, Alkharousi SJ. Psychometric properties of the Arabic self-report version of the strengths and difficulties questionnaire. Res Dev Disabil. 2016;59:211-20. [PubMed ID: 27636156]. https://doi.org/10.1016/j.ridd.2016.09.002.
-
12.
Di Riso D, Salcuni S, Chessa D, Raudino A, Lis A, Altoè G. The Strengths and Difficulties Questionnaire (SDQ). Early evidence of its reliability and validity in a community sample of Italian children. Pers Individ Differ. 2010;49(6):570-5. https://doi.org/10.1016/j.paid.2010.05.005.
-
13.
Shibata Y, Okada K, Fukumoto R, Nomura K. Psychometric properties of the parent and teacher forms of the Japanese version of the Strengths and Difficulties Questionnaire. Brain Dev. 2015;37(5):501-7. [PubMed ID: 25172302]. https://doi.org/10.1016/j.braindev.2014.08.001.
-
14.
Ortuno-Sierra J, Fonseca-Pedrero E, Paino M, i Riba SS, Muñiz J. Screening mental health problems during adolescence: Psychometric properties of the Spanish version of the Strengths and Difficulties Questionnaire. J Adolesc. 2015;38:49-56. [PubMed ID: 25460680]. https://doi.org/10.1016/j.adolescence.2014.11.001.
-
15.
Erhart M, Wetzel RM, Krügel A, Ravens-Sieberer U. Effects of phone versus mail survey methods on the measurement of health-related quality of life and emotional and behavioural problems in adolescents. BMC Public Health. 2009;9(1):491. [PubMed ID: 20042099]. [PubMed Central ID: PMC2809066]. https://doi.org/10.1186/1471-2458-9-491.
-
16.
Palmieri PA, Smith GC. Examining the structural validity of the Strengths and Difficulties Questionnaire (SDQ) in a US sample of custodial grandmothers. Psychol Assess. 2007;19(2):189-98. [PubMed ID: 17563200]. [PubMed Central ID: PMC1997309]. https://doi.org/10.1037/10403590.19.2.189.
-
17.
Björnsdotter A, Enebrink P, Ghaderi A. Psychometric properties of online administered parental strengths and difficulties questionnaire (SDQ), and normative data based on combined online and paper-and-pencil administration. Child Adolesc Psychiatry Ment Health. 2013;7(1):40. [PubMed ID: 24325882]. [PubMed Central ID: PMC3898053]. https://doi.org/10.1186/1753-2000-7-40.
-
18.
Brustad M, Skeie G, Braaten T, Slimani N, Lund E. Comparison of telephone vs face-to-face interviews in the assessment of dietary intake by the 24 h recall EPIC SOFT program—the Norwegian calibration study. Eur J Clin Nutr. 2003;57(1):107-13. https://doi.org/10.1038/sj.ejcn.1601498.
-
19.
Fenig S, Levav I, Kohn R, Yelin N. Telephone vs face-to-face interviewing in a community psychiatric survey. Am J Public Health. 1993;83(6):896-8. [PubMed ID: 8498632]. [PubMed Central ID: PMC1694719]. https://doi.org/10.2105/ajph.83.6.896.
-
20.
Swingler GH, Zwarenstein M. Telephone follow-up in a randomized controlled trial in a less developed country: feasibility, validity and representativeness. J Clin Epidemiol. 2000;53(3):331-4. [PubMed ID: 10760645]. https://doi.org/10.1016/s0895-4356(99)00166-3.
-
21.
Fukkink RG, Hermanns J. Children’s experiences with chat support and telephone support. J Child Psychol Psychiatry. 2009;50(6):759-66. [PubMed ID: 19207634]. https://doi.org/10.1111/j.1469-7610.2008.02024.x.
-
22.
Biebl SJ, DiLalla LF, Davis EK, Lynch KA, Shinn SO. Longitudinal associations among peer victimization and physical and mental health problems. J Pediatr Psychol. 2011;36(8):868-77. https://doi.org/10.1093/jpepsy/jsr025.
-
23.
Alavi A, Mohammadi MR, Mahmoudi J, Tehranidoost M, Shahrivar Z, Saadat S. The Farsi Version of the Strength and Difficulties Questionnaire Self Report Form: The Normative Data and Scale Properties. Iran J Child Neurol. 2009;3(1):27-34.
-
24.
Brandt J, Spencer M, Folstein M. The telephone interview for cognitive status. Cogn Behav Neurol. 1988;1(2):111-8.
-
25.
Hajebi A, Motevalian A, Amin-Esmaeili M, Hefazi M, Radgoodarzi R, Rahimi-Movaghar A, et al. Telephone versus face-to-face administration of the Structured Clinical Interview for Diagnostic and Statistical Manual of Mental Disorders, for diagnosis of psychotic disorders. Compr Psychiatry. 2012;53(5):579-83. [PubMed ID: 21820651]. https://doi.org/10.1016/j.comppsych.2011.06.001.
-
26.
Petrill SA, Rempell J, Oliver B, Plomin R. Testing cognitive abilities by telephone in a sample of 6-to 8-year-olds. Intelligence. 2002;30(4):353-60. https://doi.org/10.1016/S0160-2896(02)00087-9.
-
27.
Trouton A, Spinath FM, Plomin R. Twins early development study (TEDS): a multivariate, longitudinal genetic investigation of language, cognition and behavior problems in childhood. Twin Res. 2002;5(5):444-8. [PubMed ID: 12537874]. https://doi.org/10.1375/136905202320906255.