Concurrent Validity of the Behavior Rating Inventory of Executive Function in Children With Attention Deficit Hyperactivity Disorder

1. Background

Executive functions (EFs) are referred to a group of cognitive abilities responsible for goal-directed activities. They inhibit inappropriate behaviors and help us to respond to new situations by shifting our mind quickly and easily from one subject to another. These functions are considered as important factors in cognitive development of children. In fact, they are the main part of all social and personal activities (1). Observing executive dysfunction in patients with brain damage has suggested the prefrontal lobe as the central part of the brain in management of executive functioning (1). Besides, presence of executive dysfunction in many psychiatric disorders such as attention deficit hyperactivity disorder (ADHD) has become an important field of research in child and adolescent psychiatry (2).

ADHD is a common psychiatric disorder characterized by hyperactivity, impulsivity and inattentiveness, inappropriate to child’s age. Children with ADHD can be easily distracted by any stimulus and cannot sustain their attention for a long period on a subject (3). ADHD is not a simple deficit in attention, but a developmental impairment in a broad spectrum of EFs, which needs to be assessed carefully (4). For many years, assessment of EF in children was based on psychiatric interview and behavioral scales and questionnaires, which were not designed specifically for children and could not capture all aspects of executive abilities (2, 5).

Although many neuropsychological tests were developed to assess EF using direct measures of attention, mental flexibility and inhibitory control that do not rely on parent report, they are usually expensive, time-consuming and need experienced professional work. Therefore Gioia et al. (6) designed the Behavior Rating Inventory of Executive Function (BRIEF) to evaluate EF in 5-18 year-old children. Through its behavioral aspects, BRIEF makes it easy to assess existence and quality of EFs by accomplishment of a pen and paper questionnaire. From the early appearance of BRIEF, many researchers tried to compare its reliability, validity and correlation with other standardized instruments in ADHD and other psychiatric impairments such as Tourette syndrome and autism spectrum disorder (4, 7, 8).

2. Objectives

According to their results, diagnostic groups got significant higher scores in all subscales of BRIEF than the normal group. This is the first study in Iran to assess the validity of BRIEF in a group of children with ADHD compared with normal children.

3. Patients and Methods

3.1. Participants

Thirty children aged 7 to 12 years referred to the child and adolescent psychiatry clinic at Roozbeh hospital in Tehran were recruited using the simple convenient sampling method. All the participants were diagnosed as having ADHD-combined type was diagnosed in all children by a board certified child and adolescent psychiatrist based on DSM-IV-TR criteria. They did not have any history of major psychiatric disorders such as oppositional defiant or conduct disorder. All of them were drug naïve as well. The control group included 30 volunteers selected randomly from students of two elementary schools in Tehran, a female school and a male school. The controls matched for their ages, genders and IQs (based on the scores of the Raven IQ test) with the ADHD group. They did not have any history of psychiatric disorders or referral to mental health services. According to the teacher and parents’ Conners’ rating scale, they did not have the diagnosis of ADHD.

3.2. Procedure

The study was explained for parents and written consents were obtained. It was also conducted with full approval of the Institutional Review Board (IRB). The Kiddie Schedule for Affective Disorders and Schizophrenia, Present and Lifetime version, Persian version (KSADS-PL-P), was performed for all participants by a fellow of child and adolescent psychiatry to evaluate other psychiatric disorders. The Raven Progressive Matrices test was performed to assess IQ in the two groups. To eliminate any bias, IQ tests were performed between 2-4 pm. Then parents filled three questionnaires for their children; the ADHD rating scale (ADHD-RS), the Conners’ parent rating scale revised-short version (CPRS) and the BRIEF Parent form.

3.3. Rating Scales

3.3.1. BRIEF

The BRIEF (6) is an 86-item questionnaire to assess the EFs in girls and boys of 5-18 years old. There are two rating forms; teacher and parent questionnaire. Parents with at least a fifth grade education can answer these questions easily. BRIEF assesses eight non-overlapping subscales, inhibition, shifting and emotional control that result in an index; Behavioral Regulation Index (BRI), and initiation, working memory, monitoring, planning/organizing and organization of materials which are combined to make Meta Cognition Index (MCI), and a total score; “Global Executive Composite” made by these two indices. The higher the score is, the more dysfunction exists. BRIEF is developed for a wide spectrum of children with developmental and acquired neurological disorders such as learning disabilities, ADHD, traumatic brain injury, low birth weight, Tourette syndrome and pervasive developmental disorder/autism. It also contains two validity subscales of negativity and inconsistency to detect bias associated with rating scales. BRIEF has gotten both convergent and divergent validity, as well as high internal consistency (α = 0.8-0.98) and the test-retest reliability which was 0.82 and 0.88 for parent and teacher forms, respectively. There were also moderate correlations between teacher and parent ratings (r = 0.32-0.34) (6).

3.3.2. ADHD Rating Scale IV (ADHD-RS-IV)-Home Version

The ADHD-RS-IV (9) is an 18-item questionnaire, developed specifically to screen ADHD children and adolescents between 5 and 18 years old. There are two questionnaires; home and school versions, which reveal child’s behavior over the past six months at home and school. It consists of two subscales; inattention (9 items) and hyperactivity/impulsivity (9 items). Thus, we would have three raw scores at the end; 1) inattention score, 2) hyperactivity/impulsivity score and a total score by adding them together.

The scale has acceptable levels of inter-rater reliability, test-retest reliability, internal consistency, convergent validity, discriminant validity and responsiveness (10). The psychometric properties of the Farsi version of ADHD-RS-IV has been reported by Ghanizadeh previously (11).

3.3.3. Conners’ Parent Rating Scale-Revised: Short Version (CPRS-RS)

The Revised Conners’ Rating Scale (12) has three types; parent, teacher and self-report. Each has a short and a long version. CPRS-RS contains 27 items, introducing four subscales; 1) oppositional subscale, 2) inattention/cognitive Problem, 3) hyperactive/impulsive and 4) ADHD Index. The items are based on DSM-IV criteria of ADHD, so it can be used to confirm the diagnosis of ADHD.

Psychometric properties of the revised scale appear adequate as demonstrated by good internal reliability coefficients, high test-retest reliability and effective discriminative power (13). Tehrani Doost et al. found that psychometric properties of the CPRS were good in an Iranian clinical population. They suggested that CPRS was a useful instrument to discriminate children with ADHD from typically developing individuals (14).

3.3.4. K-SADS-PL

The K-SADS-PL (15) is a semi-structured interview to assess psychiatric diagnosis in children and adolescents aged 6-18 years. It assesses present and lifetime status of psychiatric disorders as well as the severity of symptoms. The K-SADS-PL is introduced from K-SADS-P according to the fourth edition of DSM and is capable of generating 32 DSM-III-R and DSM-IV Axis I child and adolescent psychiatric disorders. Diagnoses are made as definite, probable (greater than or equal to 75% of symptoms criteria met) or not present. The K-SADS-PL has excellent inter-rater reliability and results comparable to semi-structured and fully structured child diagnostic interviews. The test-retest reliability kappa coefficients are in the excellent range for present and lifetime diagnosis of major psychiatric disorders (15). The Persian version of the K-SADS-PL (K-SADS-PL-P) has been validated by Shahrivar et al. (16) in Iran. Based on their study, specificities were more than 81% for all disorders and sensitivities for most major diagnosis were between 75% and 100%. The Kappa agreements for most diagnoses were higher than 0.4 and the test-retest reliabilities were between 0.38 and 0.87.

3.3.5. Raven’s Standard Progressive Matrices

The Raven’s Standard Progressive Matrices (SPM) is a standard measure for non-verbal intelligence and perceptual reasoning. It was used as the intelligence test to match the IQs of the two groups (17). SPM has been validated and standardized for 5-18-year old children and adolescents in Iran (18).

3.4. Statistical Analysis

Statistical analysis was performed using SPSS 15 (Softonic International, S.A.). The prevalence of subscales and indices were calculated by descriptive statistics. T-independent test was performed to compare the differences between two groups of variables. The Pearson correlation test was performed to find the correlation between the variables of the questionnaires.

4. Results

As can be seen in Table 1, thirty children in each group were enrolled (6 girls and 24 boys). The mean ages were 8.96 years (SD = 1.5) in ADHD group and 9.03 (SD = 1.4) in control group. The IQ means were 110.86 (SD = 10.77) and 115.33 (SD = 7.07) in ADHD and control groups, respectively. There were no significant differences between the two groups regarding their age, gender and IQ.

4.1. Group Comparison on BRIEF Subscales and Indices

The means of all eight subscales and two indices of BRIEF are summarized in Table 2. It shows significant differences between ADHD and control groups for all BRIEF subscales (P < 0.001).

4.2. Group Comparison on ADHD-RS

As shown in Table 3, in both inattention and hyperactivity subscales, the differences between the two groups were significant.

4.3. Group Comparison on CPRS-RS

Scores of all four subscales of CPRS-RS are listed in Table 4. Just like the two other questionnaires, the differences were significant between the two groups.

4.4. Correlations Between BRIEF and ADHD-RS-IV

Correlations between BRIEF and ADHD-RS are presented in Table 5. In ADHD group, most BRIEF variables were significantly correlated with ADHD-RS variables (P < 0.05). In both groups MCI was much more correlated with ADHD-RS Inattentive symptoms (r = 0.723 in ADHD group, r = 0.739 in control group) than with hyperactivity symptoms (r = 0.560 in ADHD group, r = 0.259 in control group). On the contrary, BRI was strongly correlated with ADHD-RS hyperactivity symptoms (r = 0.591 in ADHD group, r = 0.549 in control group) than with Inattentive symptoms (r = 0.437 in ADHD group, r = 0.548 in control group).

4.5. Correlations Between BRIEF and CPRS-RS

Correlations between BRIEF and CPRS-RS are shown in Table 6. In both groups, most BRIEF variables were correlated with CPRS variables significantly. As can be seen, similar to the pattern of Table 3, the BRIEF MCI was more correlated with CPRS-RS inattentive symptoms (r = 0.621 in ADHD group, r = 0.730 in control group) than hyperactivity symptoms (r = 0.560 in ADHD group, r = -0.021 in control group). Considering BRI, although it was strongly correlated with CPRS-R: S Oppositional symptoms (r = 0.778 in ADHD group, r = 0.523 in control group), there was a significant correlation between BRI and hyperactivity symptoms (r = 0.585 in ADHD group, r = 0.387 in control group).

4.6. Age Effects

According to linear regression, BRIEF scores were not influenced by children’s age in any group (P > 0.05).

5. Discussion

This study was designed to evaluate the validity of parent rating of BRIEF in children with ADHD. Therefore, as the first step, we intentionally eliminated referees who had another disorder in addition to ADHD, since EDF can be observed in many psychiatric disorders. Our findings strongly indicated that BRIEF could differentiate children with ADHD from normal group; executive dysfunction of these children could be reflected on BRIEF questionnaire.

The parent ratings of BRIEF were significantly correlated with two other parents’ ratings, which are used to evaluate ADHD symptoms in children and adolescents. This finding supports the concurrent validity of the parent form of BRIEF, which has been reported by others (4, 19, 20). The patterns of correlation between BRIEF subscales and indices with CPRS and ADHD-RS suggest discriminative validity of the BRIEF rating to compare different types of ADHD. Working memory is theoretically based on attentional functioning, so the BRIEF working memory subscale is expected to be correlated with attention scales of other rating scales. Likewise, the BRIEF inhibit subscale should correlate with measures of externalizing behaviors. The results of our study showed that the BRIEF MCI, especially working memory has a high degree of correlation with ADHD-RS and CPRS inattention scales. Conversely, the BRIEF BRI particularly inhibition is strongly correlated with hyperactivity indices of the two other parent ratings. These results are consistent with the findings of multiple studies (4, 8, 21), although Sullivan and Riccio (22) suggested that the ability of BRIEF to distinguish ADHD and different clinical groups deserves further investigations.

The BRIEF considers behavioral patterns, which are compatible with ADHD symptoms, so it can be used to find the symptoms of basic behavioral dysregulation such as sustained attention and inhibition of competing responses. The items of BRIEF can help parents and teachers to better understand the nature of behavioral criteria of ADHD in their kids. Another finding of our study was lower mean scores of all subscales and indices of BRIEF in our control group than the normal children reported in the manual of BRIEF. It would be due to some methodological or probable cross-cultural differences and suggests further investigations.

Our study had some limitations. First, the small sample size and the narrow age range of the participants limit generalizability of the results. Secondly, we did not consider teachers’ ratings on ADHD symptoms, an informative pool of data, which could be quite helpful. Third, as all participants with ADHD had its combined type, we could not evaluate the validity of BRIEF to differentiate ADHD subtypes. In conclusion, the BRIEF can be considered as a valid tool to discriminate children with ADHD from typically developing children referred to psychiatric clinics.