A Differential Item Functioning (DIF) Study of the Infection Control Standard Isolation Precaution Instrument Across Gender and Major Among Healthcare Workers

1. Background

Healthcare-associated infections (HAIs) are a major health problem that can involve patients, visitors, or any healthcare workers (HCWs) in hospitals or other healthcare facilities and are considered a significant concern for the general public (1, 2). According to the Centers for Disease Control and Prevention (CDC), standard isolation precautions that have recommendations on topics including hand hygiene, personal protective equipment (PPE), respiratory hygiene/cough etiquette, patient placement, patient-care equipment, and instruments/devices, care for the environment, textiles and laundry, safe injection practices, infection control practices for special lumbar puncture procedures, and worker safety are the most critical and necessary infection control measures. If these measures are observed strictly by HCWs, the spread of microorganisms can be significantly reduced (3, 4).

Lack of knowledge, forgetfulness, shortage of time, limited resources, insufficient support by managers, and HCWs’ socio-demographic characteristics, including gender, age, working site, experience, job category, and marital status, could affect HCWs’ compliance with standard precautions (5). Measuring HCWs’ knowledge, attitudes, and practices (KAPs) regarding infection control standard isolation precautions needs a valid and reliable instrument. A questionnaire is a common tool for data collection. The primary purpose of a questionnaire is to collect accurate (i.e., valid) and consistent (i.e., reliable) data (6). There is a paucity of credible instruments assessing HCWs’ KAPs regarding infection control standard isolation precautions, especially in Iran. In an attempt to address this gap, Askarian et al. (7) developed a questionnaire for assessing HCWs’ KAPs regarding infection control standard isolation precautions. Using a sample of 622 medical students, they found Cronbach’s Alphas of 0.726, 0.765, and 0.782 for the knowledge, practice, and attitude tests, respectively. The content validity of the instrument was established by a group of experts comprising of infection control experts, medical experts, and a psychiatrist. It was also reviewed by experts from the Iranian national expert group of infection control specialists. In 2006, they also repeated a similar study on a sample of physicians, surgeons, surgical residents, and medical residents. Findings showed that, generally, median scores for knowledge and attitudes were moderate to high. Surgeons were the only group that revealed a moderate to strong (r = 0.748, P-value < 0.001) relationship between knowledge and attitudes, while for other medical groups, this relationship was weak. The mentioned study also showed that more than 80% of all medical practitioners had not received previous education on infection control standard isolation precautions, and more than 80% were willing to be trained (8). Finally, in 2007, Askarian et al. (9) administered the same questionnaires on a sample of nursing, assistant nursing, and midwifery practitioners and students, along with interviews. Their findings revealed that about 91% of participants needed additional infection control education, especially on standard isolation precautions. They found positive correlations between KAPs for nurses, assistant nurses, and midwifery instructors and students.

Based on previous studies, this instrument shows high promises for assessing HCWs’ KAPs regarding infection control standard isolation precautions. KAPs regarding standard precautions among HCWs are limited, and previous studies have shown a variety of KAPs among different groups of HCWs (10-12). As part of the instrument validation, it is important to ensure the uniformity of assessment across different demographic groups. Such a study is known as measurement invariance (MI) (13). Technically, it is important to test whether the probability of responding to a specific item differs across different identifiable groups after controlling for the construct being measured (13). This is known as testing differential item functioning (DIF). Generally speaking, there are two types of DIF. The uniform DIF implies constant differences in the probability of responding to an item between groups along the continuum of respondent’s knowledge or ability (14). Figure 1 shows an example of the uniform DIF.

An example of the uniform DIF across two groups

Figure 1.

An example of the uniform DIF across two groups

Another type of DIF is known as the non-uniform or crossing DIF. A non-uniform DIF implies that the differences in the probability of responding to an item not only depend on the respondent’s group membership but also depend on his/her knowledge or ability. In another word, there’s an interaction between group membership and knowledge or ability. Figure 2 shows an example of the non-uniform DIF.

An example of the non-uniform DIF across two groups

Figure 2.

An example of the non-uniform DIF across two groups

The best-case scenario is not observing the DIF. This means that the probability of correctly answering a question should only be dependent on the respondent’s knowledge or ability, not on other external variables such as gender, social status, job status, or race. Ensuring MI or the absence of DIF at the item level will result in comparable scores across demographic groups (15). Lack of MI at the item level (or the presence of DIF) may result in imprecise group differences in the observed scores. MI evaluates the equivalency of a construct across groups or measurement occasions. MI is relevant to group comparisons (e.g., the analysis of variance (ANOVA)), comparing means across repeated measures (e.g., pretest-posttest designs), and comparing the relationships between constructs across groups. Measurement non-invariance (MNI), on the other hand, implies that the construct of interest is not comparable across groups or occasions. This means that, under lack of MI, simply using the observed scores for statistical analysis such as ANOVA would yield incorrect conclusions because it is unclear whether the observed differences between the groups are real differences or are due to different perceptions of individuals. Another occasion in which MNIs can generate incorrect conclusions is when assessing the effectiveness of interventions. Such studies usually involve pretest-posttest measurements, control-treatment groups, or both. Researchers should ensure that pretest-posttest scores or the scores of control-treatment groups yield similar meanings and psychometric properties. If pretest-posttest scores or the scores of control-treatment groups are non-invariant, then it will not be clear if an observed change is due to the intervention or a change in participant’s perceptions of the construct under study. This emphasizes the importance of establishing MI before using the observed score for any statistical analysis (16).

2. Objectives

This study aimed to assess DIF across gender and major (medical students versus nursing practitioners) for the knowledge and practice items as, according to previous studies, these two tests have a similar way of scoring (i.e., correct/incorrect) and are more relevant for practical and educational purposes.

3. Methods

3.1. Participants and Data

A total of 1070 participants responded to the questionnaire. The sample consisted of 306 (28.6%) males and 764 (71.4%) females. There were 466 (41.7%) medical students and 624 (58.3%) nursing practitioners. The mean age of medical students was 27.32 ± 5.29, and that of nursing practitioners was 29.72 ± 6.21 years. The mean score for the knowledge subscale was 8.23 ± 1.2, while that for the practice subscale was 4.13 ± 2.95.

The ethical approval codes were obtained by the Ethical Committee of Shiraz University of Medical Sciences (code: IR.sums.med.rec.398.315). Verbal informed consent was obtained from all participants.

3.2. Instrument

The instrument had three subscales (including knowledge, practice, and attitude) with nine questions for each subscale. The questions related to the knowledge subscale had three possible answers (yes, no, and I don’t know). All responses were scored as 1 for the correct answer and 0 for other answers. Therefore, the score for the knowledge test ranged between zero (no correct answer) and nine (all answers are correct). The questions assessing the practice subscale had five possible answers (always; often; sometimes; seldom; and never). A score of 1was allocated to ‘always’ and 0 to other answers. The total score ranged between zero and nine (all questions answered with ‘always’). The goal of this subscale is to assess the extent to which HCWs practice the standards. In the present study, the reliability index using Cronbach’s Alpha was 0.602 and 0.892 for the knowledge and practice subscales, respectively.

3.3. Data Analysis

The descriptive statistics and estimation of the reliability index were performed using IBM SPSS software, version 22. The DIF and DTF analyses were performed using the DIFAS 5.0 software (17)). To detect DIF, the Mantel-Haenszel chi-square (MH CHI) statistic (18, 19), the Mantel-Haenszel common log-odds ratio (MH LOR) (20), and the standardized Mantel-Haenszel log-odds ratio (SMH LORZ) were used. The MH CHI was distributed with chi-square with one degree of freedom, so the values greater than 3.84 were considered statistically significant at α = 0.05. Positive values of the MH LOR indicated a DIF level in favor of the reference group, and negative values indicated a DIF level in favor of the focal groups. For the SMH LORZ, values higher than 2.0 or lower than -2.0 might denote the evidence for the presence of DIF. Furthermore, to classify the items showing DIF levels as small (A), moderate (B), and large (C), the ETS (Educational Testing Services) categorization scheme (21) was used. These categories were based on the differences calculated on the scale of item difficulty, known as Mantel-Haenszel delta difference (MH D-DIF). A statistically non-significant MH D-DIF of lower than 1.0 in an absolute value would categorize an item as A (i.e., small DIF). Items with MH D-DIF, which were significantly higher than 1.0 and an absolute value of 1.5 or higher would be categorized as C (i.e., large DIF). All other items would be categorized as B (i.e., medium DIF) (18). Given that all items were dichotomous, the variance of DIF across items (i.e., τ²) was used to detect DTF. DTF compares a set of items across groups. In other words, it evaluates the comparability of total test scores (i.e., the observed scores) across groups. In this sense, it is an extension of DIF (22). As proposed by Penfield and Algina, values of τ² < 0.07 indicate small, 0.07 ≤ τ² ≤ 0.14 indicate medium, and τ² > 0.14 indicate large DTF (23). Two demographic characteristics were chosen for detecting DIF and DTF: Gender (the reference group = male, versus the focal group = female) and major (the reference group = medical students, versus the focal group = nursing practitioners).

4. Results

The results of DIF and DTF are reported in Tables 1 and 2. In assessing DIF and DTF across genders, the male category was set as the reference group, and regarding major, the medical category was set as the reference group.

A review of DIF values for the knowledge subscale (Table 1) shows that the majority of the items did not exhibit DIF across genders. Only item 2, “Washing hands before and after using gloves” (in favor of males), and item 8, “Bending needles before disposal” (in favor of females), showed moderate levels of DIF with significant MH CHI and a moderate level of DIF. At the scale level, a very small value of τ² was found, suggesting that the overall effects of the DIF items compensate each other, resulting in a comparable total score across genders. As for the medical group versus the nursing group contrast, two items showed significant MH CHIs with moderate DIF levels (i.e., item 6, “Washing hands with betadine after contact with patients during procedures and activities that are likely to generate splashes or sprays of blood and body fluids”, in favor of the medical group and item 9, “Wearing a gown during procedures that are likely to generate splashes or sprays of blood and body fluids”, in favor of the nursing group). Additionally, two items showed significant MH CHIs with large DIF levels. Item 1, “Washing hands before and after patient care”, showed a large DIF level in favor of the medical group, indicating that members in this group had a significantly higher chance of correctly answering this question than those in the nursing group. In other words, just being in this group significantly increased their chance for a correct answer, and not necessarily their level of knowledge. On the contrary, item 8, “Bending needles before disposal”, showed a large DIF level in favor of the nursing group. This means that being in this group significantly increased their chance of correctly answering this question. A value of τ² = 0.515 indicated a large effect at the scale level, leading to incomparable total scores across majors.

As it is evident from the values in Table 2, all items of the practice subscale function similarly across genders, except for item 7, “Wearing a surgical mask to protect nose and mouth during procedures and activities that are likely to generate splashes or sprays of blood and body fluids”, that showed a significant MH CHI with moderate DIF in favor of male respondents. Similar to the knowledge subscale, a very small value of τ² suggested a comparable total score across genders. The practice subscale showed more items with significant MH CHIs and moderate to large DIF levels across majors. Item 2 and item 5, “Wearing goggles to protect mucous membranes of the eyes during procedures that are likely to generate splashes or sprays of blood and body fluids”, showed moderate levels of DIF, both in favor of the nursing group. Three items revealed large levels of DIF. Item 4, “Wearing gloves before touching mucous membranes and non-intact skin”, and item 7 showed large DIF levels in favor of medical respondents, and item 9 showed large DIF in favor of nurse respondents. A value of τ² = 0.754 indicated a large effect at the scale level, leading to incomparable total scores across majors.

5. Discussion

Our findings demonstrated very similar DIF outcomes for the knowledge and practice subscales with one or two items showing moderate DIF, but comparable total scores, across genders. Small or very small values of τ² implied that the observed DIF levels at the item level either were negligible or the items displaying DIF in favor of different groups nullified each other at the test level. Several factors can contribute to the DIF. For example, item 8, “Bending needles before disposal”, showed moderate DIF in favor of female respondents and large DIF in favor of nurse respondents on the knowledge subscale. One possible reason could be the fact that nursing practitioners usually administer injections and, therefore, are responsible for the safe disposal of needles and many nurses are female. In contrast, item 7, “Wearing a surgical mask to protect nose and mouth during procedures and activities that are likely to generate splashes or sprays of blood and body fluids”, showed moderate DIF in the practice subscale in favor of male respondents and large DIF in favor of the medical group. One potential explanation could be the fact that most splash generating procedures such as suturing, endotracheal suctioning, bronchoscopy, and invasive vascular procedures that need close distance to patients are administered by medical doctors in Iranian hospitals.

Item 9, “Wearing a gown during procedures that are likely to generate splashes or sprays of blood and body fluids”, on the practice subscale showing large DIF in favor of nurses could be explained by the fact that these procedures (such as care for patients), with the possibility of an extensive splash of blood, body fluids, secretions, or excretions, are usually performed by nurses, so they know more about these types of procedures. Similarly, item 4, “Wearing gloves before touching mucous membranes and non-intact skin”, on the practice subscale showing large DIF in favor of the medical group, could be because of the reason that most procedures need touching mucous membranes and non-intact skin which are performed by medical doctors in Iran.

This study aimed to examine DIF and DTF of the knowledge and practice subscales of the KAPs regarding the infection control standard isolation precautions instrument. Our findings indicated large DIF and DTF levels when the instrument was administered to the respondents from the medical and nursing groups. There could be several explanations for these findings, some of which will be discussed in the same section. This was an important step towards ensuring and enhancing the validity of this instrument. Our findings highlighted the fact that more attention should be paid when developing the items. The majority of the items, that exhibited large DIF in favor of a particular group, had contents or examples most familiar or relevant to that group. This adds unwanted irrelevant variance to the data, biasing the results of any statistical analyses. In terms of test fairness, we wanted our instruments to measure the construct of interest (e.g., the level of practicing infection control standard isolation precautions) regardless of respondents’ demographic backgrounds, as these standards are set to be exercised by all HCWs. Fortunately, DIF and DTF levels were negligible across genders, but the revision of the items’ contents when it comes to practice is needed. We hope that this study sets an example and shows the importance of paying attention to valid evidence for instruments developed within the field of healthcare.