1. Background
Digital epidemiology is a novel and rapidly evolving subfield of epidemiology that leverages digital data streams and advanced computational methods to understand and control the spread of diseases (1). Unlike traditional epidemiology, which primarily relies on structured health data collected through clinical settings, surveillance systems, and field investigations, digital epidemiology uses data generated outside conventional healthcare systems. These sources include social media platforms, internet search queries, mobile phone data, wearable devices, and other digital footprints left by individuals during daily activities.
The advent of big data technologies and machine learning algorithms has substantially enhanced the capacity of public health professionals to analyze vast and complex datasets in near real time. This capability enables earlier detection of disease outbreaks, more precise modeling of disease dynamics, and improved understanding of population health-related behaviors (2). For example, digital epidemiology tools have been used to monitor influenza-like illnesses through Twitter posts, track the spread of COVID-19 by analyzing mobility data, and detect adverse drug reactions from online forums (3).
Although digital epidemiology offers considerable potential, it also presents several challenges, including data privacy concerns, issues related to data representativeness, and the need for specialized analytical skills. Therefore, it is imperative that the next generation of epidemiologists and public health practitioners be equipped with the knowledge, attitudes, and practical skills necessary to effectively apply digital epidemiology methods (4).
Despite its growing importance, there is a notable lack of validated instruments to assess the understanding and application of digital epidemiology concepts, particularly in countries such as Iran, where the field is still developing (3). Developing a robust, reliable, and valid questionnaire to measure KAP regarding digital epidemiology can help identify educational needs, inform curriculum design, and ultimately facilitate the integration of digital tools into public health initiatives (5). In the present study, awareness is conceptualized as self-perceived familiarity with and understanding of digital epidemiology concepts, reflecting participants’ confidence and readiness to apply knowledge in practice. Although objective knowledge is important, self-perceived awareness provides insight into participants’ perceived preparedness to engage in digital epidemiology tasks and is consistent with prior KAP studies (6, 7).
2. Objectives
This study aimed to address this gap by developing and validating a questionnaire tailored to students and graduates in epidemiology and public health in Iran. Using rigorous qualitative and quantitative methods, the study sought to establish the psychometric soundness of the instrument, thereby providing a valuable resource for educational assessment and future research in digital epidemiology.
3. Methods
3.1. Study Design and Participants
This study employed a sequential exploratory mixed-methods design comprising qualitative and quantitative phases. The target population for questionnaire development and psychometric evaluation included students and graduates of epidemiology and public health programs in 2023. The study protocol was reviewed and approved by the Ethics Committee of Shahrekord University of Medical Sciences (ethics code: IR.SKUMS.REC.1401.054).
3.2. Phase 1: Qualitative Study for Questionnaire Development
In the first phase, a qualitative approach was used, including a comprehensive literature review and expert panel discussions. Twenty experts in epidemiology participated in a focus group to identify relevant items related to 3 constructs: awareness, attitude, and practice. Based on these inputs, an initial draft of the questionnaire was developed.
3.3. Content Validity Assessment
To ensure content validity, the preliminary questionnaire items generated from the literature review were evaluated by an expert panel comprising 20 specialists in epidemiology and public health with experience in digital health and epidemiological research.
Experts were asked to assess each item in terms of necessity, relevance, clarity, and simplicity. For quantitative evaluation, the Content Validity Ratio (CVR) and Content Validity Index (CVI) were calculated.
For the CVR assessment, experts rated each item as essential, useful but not essential, or not essential. CVR values were calculated using Lawshe’s method. According to Lawshe’s table, for a panel of 20 experts, items with CVR values below 0.42 were considered unacceptable and were revised or removed.
For the CVI assessment, experts rated item relevance on a 4-point scale. Items with CVI values below 0.79 were revised or removed, whereas items with CVI values above this threshold were retained.
Based on expert feedback and quantitative indices, several items were reworded or removed to improve clarity and conceptual alignment. After this process, the refined questionnaire was finalized for the quantitative phase.
In the present study, awareness was conceptualized as self-perceived familiarity with and understanding of digital epidemiology concepts, reflecting participants’ confidence and readiness to apply this knowledge in practice. Although objective knowledge is important, self-perceived awareness provides insight into participants’ perceived preparedness to engage in digital epidemiology tasks and is consistent with prior KAP studies (6, 7).
3.4. Phase 2: Quantitative Cross-Sectional Study
The second phase comprised a web-based cross-sectional survey to assess the awareness, attitudes, and practices of students and graduates in epidemiology and public health regarding digital epidemiology in Iran. Data collection was conducted using the preliminary questionnaire designed in phase 1.
3.5. Data Collection Instrument and Measurement
The primary data collection instrument was a self-constructed questionnaire comprising 2 sections: demographic information (gender, age, education level, marital status, and employment) and the main items assessing the constructs of awareness, attitude, and practice. The main items were measured on a 5-point Likert scale ranging from strongly disagree to strongly agree.
3.6. Reliability and Validity Assessment
To evaluate the psychometric properties of the questionnaire, data from 200 participants were analyzed using SMART-PLS version 3.2.4. Given the sample size and the non-normality of the data distribution, confirmatory factor analysis (CFA) was performed using a second-order factor model to assess the reliability and validity of the instrument. The analysis included evaluation of internal consistency, convergent validity, and discriminant validity. A second-order factor model was used to reflect the multidimensional nature of the constructs of awareness, attitude, and practice. Each construct comprised several subdimensions that captured distinct aspects of the overarching concept. The use of a second-order model enabled the summarization of these subdimensions into a single latent construct, reduced model complexity, improved interpretability, and captured variance at both the subdimension and overall construct levels.
4. Results
The mean age of the participants was 36.30 years. Among them, 51% were married. Regarding educational status, the largest proportion (45%) held a doctorate degree, while 32% were students. In terms of occupational status, 26% of the participants were faculty members, and 32% were healthcare staff (Table 1).
Table 1.Demographic Characteristics of the Study Participants a
| Variables | Values |
|---|---|
| Gender | |
| Male | 102 (51) |
| Female | 48 (49) |
| Marital status | |
| Single | 78 (39) |
| Married | 122 (61) |
| Education | |
| MD | 8 (4) |
| Bachelor’s degree | 20 (10.2) |
| Master’s degree | 78 (39.8) |
| PhD | 90 (45.9) |
| Job | |
| Science Committee | 52 (26.5) |
| Health system employee | 64 (32.7) |
| University student | 68 (34.7) |
| Unemployee | 12 (6.1) |
| Age | 36.20 ± 7.78 |
a Values are expressed as No. (%) or mean ± SD.
4.1. Reliability
In this study, reliability was assessed using three methods: factor loadings, Cronbach’s alpha coefficient, and composite reliability. All factor loading coefficients for the questionnaire items were above 0.4, indicating that this criterion was met. Moreover, as the acceptable threshold for both Cronbach’s alpha and composite reliability is 0.7, the results presented in Table 2 indicate that these criteria were satisfactorily met for the latent variables. Therefore, the reliability of the study instrument can be confirmed as acceptable.
Table 2.Reliability Assessment of the Modified Model
| Dimensions and Questions | Factor Loading | Cronbach’s Alpha | Composite Reliability |
|---|---|---|---|
| Awareness | 0.876 | 0.901 | |
| S1 | 0.656 | ||
| S2 | 0.541 | ||
| S3 | 0.815 | ||
| S4 | 0.743 | ||
| S5 | 0.809 | ||
| S6 | 0.758 | ||
| S8 | 0.677 | ||
| S9 | 0.695 | ||
| S10 | 0.588 | ||
| S11 | 0.584 | ||
| Attitude | 0.791 | 0.881 | |
| N1 | 0.891 | ||
| N2 | 0.920 | ||
| N4 | 0.708 | ||
| Performance | 0.893 | 0.913 | |
| P1 | 0.660 | ||
| P2 | 0.824 | ||
| P3 | 0.791 | ||
| P4 | 0.766 | ||
| P5 | 0.661 | ||
| P6 | 0.743 | ||
| P7 | 0.674 | ||
| P8 | 0.664 | ||
| P9 | 0.657 | ||
| P10 | 0.694 |
Given that the acceptable threshold for the Average Variance Extracted (AVE) is 0.5, the results shown in Table 3 indicate that this criterion was within an acceptable range for the latent variables of practice and attitude. Although the awareness dimension was slightly below the threshold, it was still considered acceptable within some tolerance. Consequently, convergent validity was also confirmed. The SMART-PLS outputs indicated that, after minor modifications, the researcher-made questionnaire demonstrated satisfactory validity, including convergent and discriminant validity, and reliability, including standardized factor loadings, composite reliability, and Cronbach’s alpha (Tables 3 - 5).
Table 3.Convergent Validity Results of the Research Latent Variables
| Hidden Variables | Average Variance Extracted (AVE) |
|---|---|
| Awareness | 0.48 |
| Attitude | 0.51 |
| Performance | 0.71 |
Table 4.Correlation of the Construct with Its Indicators (Divergent Validity)
| Indicators | Awareness | Attitude | Performance |
|---|---|---|---|
| N1 | 0.300 | -0.045 | 0.891 |
| N2 | 0.336 | 0.022 | 0.920 |
| N4 | 0.249 | 0.084 | 0.708 |
| P1 | 0.088 | 0.660 | 0.026 |
| P10 | 0.165 | 0.694 | 0.070 |
| P2 | 0.281 | 0.824 | -0.003 |
| P3 | 0.267 | 0.791 | 0.010 |
| P4 | 0.270 | 0.766 | -0.109 |
| P5 | 0.105 | 0.661 | 0.015 |
| P6 | 0.223 | 0.743 | 0.017 |
| P7 | 0.152 | 0.674 | 0.218 |
| P8 | 0.190 | 0.664 | 0.069 |
| P9 | 0.285 | 0.657 | -0.102 |
| S1 | 0.656 | 0.124 | 0.155 |
| S10 | 0.588 | 0.247 | 0.170 |
| S11 | 0.584 | 0.155 | 0.419 |
| S2 | 0.541 | 0.154 | 0.228 |
| S3 | 0.815 | 0.250 | 0.232 |
| S4 | 0.743 | 0.216 | 0.161 |
| S5 | 0.809 | 0.245 | 0.250 |
| S6 | 0.758 | 0.204 | 0.309 |
| S8 | 0.677 | 0.126 | 0.227 |
| S9 | 0.695 | 0.264 | 0.296 |
Table 5.Divergent Validity According to the Fornell-Larcker Method
| Hidden Variables | Awareness | Attitude | Performance |
|---|---|---|---|
| Awareness | 0.96 | - | - |
| Attitude | 0.29 | 0.71 | - |
| Performance | 0.35 | 0.02 | 0.84 |
5. Discussion
The present study aimed to develop and validate a questionnaire assessing awareness, attitude, and practice regarding digital epidemiology among students and graduates of epidemiology and public health in Iran. Regarding the reliability of the instrument, given that a value of 0.7 is considered an acceptable threshold for determining both external and internal reliability (8, 9), it can be concluded that the tool demonstrates good reliability. Based on the researchers’ review, no previous study has investigated awareness, attitude, and practice toward digital epidemiology, and this questionnaire is the first instrument developed to measure this topic. The extracted factors are discussed in the following sections (9).
The present findings are consistent with prior KAP research in related domains, such as digital health literacy among healthcare professionals. Similar to these studies, participants showed generally positive attitudes toward digital technologies but exhibited moderate knowledge levels and variable engagement with practical tools. This pattern highlights a common challenge in emerging technology education: while motivation and attitudes may be favorable, gaps in knowledge and hands-on experience persist (10). In the context of digital epidemiology, our results underscore the importance of integrating targeted training on digital tools, big data applications, and online data sources into educational curricula. Addressing these gaps can enhance both understanding and practical competencies and ultimately support the effective use of digital epidemiology methods in research and public health practice (11).
Discriminant validity was assessed using both the Fornell-Larcker criterion and the Heterotrait-Monotrait (HTMT) ratio. All HTMT values were below 0.85, indicating sufficient discriminant validity among the awareness, attitude, and practice constructs. These findings support the adequacy of the measurement model in differentiating among the 3 constructs (12).
5.1. Awareness
This factor consisted of 10 items measuring the knowledge and information of students and graduates about digital epidemiology. Among the 11 designed awareness questions, the item with a factor loading below 0.4 was removed, resulting in an increase in reliability to an acceptable level. Awareness of the digital epidemiology process is a crucial component of education; before expecting an individual to perform a behavior and demonstrate desirable practice, the individual must first understand what the behavior entails and acquire information about that specific behavior. Therefore, before any action is taken, an individual’s awareness of the subject must be raised and given significant importance. A thorough search revealed no existing instrument measuring awareness in the field of digital epidemiology; hence, assessing this construct with an appropriate tool can play a vital role in understanding the knowledge and awareness of students and graduates regarding digital epidemiology (13).
The AVE value for the awareness construct was 0.48, slightly below the recommended threshold of 0.50. This may be attributed to the multidimensional nature of the construct and heterogeneity in participants’ experience with digital epidemiology. Although slightly below the threshold, the composite reliability of 0.75 indicates acceptable internal consistency. According to methodological standards, AVE values slightly below 0.50 can be tolerated if composite reliability is adequate (14). We acknowledge this as a limitation and recommend that future studies refine the awareness construct and test it in broader populations to further enhance convergent validity. Although the awareness construct demonstrated acceptable internal consistency (composite reliability = 0.75), its AVE value was slightly below the recommended threshold (0.48). Therefore, the convergent validity of the awareness construct should be interpreted with caution. Future studies are encouraged to refine the awareness items, expand testing to larger and more diverse populations, and further evaluate the construct’s psychometric properties. This cautious interpretation ensures that readers understand both the strengths and limitations of the awareness measure and underscores the importance of ongoing refinement for future research (14).
5.2. Attitude
This factor, consisting of 3 items, evaluated the attitudes of students and graduates toward digital epidemiology. Among the 4 designed attitude questions, the item with a factor loading below 0.4 was excluded, thereby increasing the reliability of this section to an acceptable level. Without a positive attitude toward the implementation and integration of digital epidemiology into disease surveillance systems and the effective use of digital epidemiology in epidemic control, desirable practice cannot be expected. Therefore, the availability of such an instrument can assist researchers in measuring this variable and fostering positive inclinations toward the adoption of digital epidemiology.
5.3. Practice
The other extracted factor, practice, consisted of 10 items reflecting the extent to which graduates and students had engaged with studies that use big data, contact tracing tools, web pages, and Google Trends to achieve study objectives. It also assessed whether participants had ever used digital media, such as the internet, mobile phones, digital paper, and digital television, for epidemiological purposes; whether they had used electronic questionnaires for data collection; participated in electronic surveys and research projects during the COVID-19 pandemic; used mobile phone data networks for health system goals; applied global positioning system data for health objectives; and employed electronic tools to conduct their research. There is a clear need for a standardized tool in this area. Therefore, assessing this construct with an appropriate instrument can significantly contribute to understanding effective practices in digital epidemiology and enable the design and evaluation of targeted interventions.
In this questionnaire, special attention was paid to the design, layout, textual format, and instructions that influence response rates and response accuracy. For the Likert scale questions, 5 response options were provided to allow respondents to select a neutral option, which helps increase reliability (9).
Although this instrument demonstrates acceptable validity and reliability, some fundamental aspects were overlooked. For example, content validity was not measured by administering the questionnaire at 2 different times to 2 different groups, which might affect the study results. Additionally, demographic characteristics were not examined in this study, and variability in these characteristics among participants could have influenced the findings (13). Furthermore, the questionnaire establishes the validity and reliability of the defined constructs, thereby enhancing its effectiveness; however, only closed-ended questions were used. Open-ended questions allow respondents to provide more extensive information and explore additional dimensions of the topic, but such data can become too broad, making it difficult to identify the core issues even if the open-ended questions have high validity. Therefore, open-ended questions were excluded from this study.
Despite the validation process and acceptable reliability observed, this study had some limitations. The exclusive use of Iranian participants limits the global generalizability of the findings. Criterion validity was not evaluated in this questionnaire. Moreover, this questionnaire is in Persian, and its application in other languages and populations requires appropriate validity and reliability testing (10).
The convergent validity results indicated that the AVE values for the knowledge and practice domains were slightly lower than the commonly recommended threshold of 0.50. This finding may be explained by several factors. Digital epidemiology is a relatively new and multidisciplinary field, and participants may differ considerably in their exposure to and experience with digital tools and data sources. Such variability can naturally lead to differences in responses to knowledge and practice items, resulting in slightly lower inter-item convergence (15).
In addition, the knowledge and practice sections were intentionally designed to capture diverse aspects of digital epidemiology, including big data utilization, digital surveillance systems, online data sources, mobile data, and electronic research tools. Although this broad coverage strengthens the content validity of the instrument, it may reduce inter-item correlations and consequently lower AVE values.
During model refinement, items with factor loadings below acceptable levels were removed, which improved construct reliability while preserving conceptual comprehensiveness. Following these modifications, composite reliability values for all constructs remained above acceptable thresholds, confirming adequate internal consistency (12).
Methodological literature suggests that AVE values slightly below 0.50 may still be acceptable when composite reliability values exceed recommended limits, as observed in the present study. Therefore, despite marginally lower AVE values in the knowledge domain, the overall reliability and validity indicators support the adequacy of the developed questionnaire. Future studies may further refine these domains to enhance construct convergence across different populations and settings (15).
5.4. Limitations
The present study employed a convenience sample of 200 students and graduates of epidemiology and public health in Iran who participated in a web-based survey. Although this approach was practical and allowed efficient data collection, it may have introduced selection bias. Respondents who chose to participate may have had greater interest, motivation, or familiarity with digital epidemiology than nonrespondents. Consequently, the findings should be interpreted with caution, and generalization to the wider population of students and graduates may be limited. Future studies using larger, randomly selected, and more diverse samples are recommended to improve the external validity and generalizability of the results.
5.5. Conclusions
This study successfully developed and validated a comprehensive questionnaire to assess KAP related to digital epidemiology among students and graduates in epidemiology and public health in Iran. Through a rigorous 2-phase mixed-methods approach, including qualitative item generation followed by quantitative psychometric evaluation, the instrument demonstrated satisfactory validity and reliability. The final questionnaire provides a robust and contextually relevant tool for measuring the critical constructs underlying digital epidemiology literacy and engagement.
The awareness domain indicated that awareness of digital epidemiology is foundational for enabling effective behavioral change, highlighting the need to strengthen educational efforts in this emerging field. The attitudinal assessment emphasized the importance of fostering positive perceptions toward the integration of digital epidemiology within health surveillance systems, as such attitudes are essential precursors to successful implementation and use. The practice domain elucidated varied levels of engagement with digital epidemiology tools and resources, indicating the need for targeted interventions to improve practical competencies.
Importantly, this instrument fills a significant gap in the current literature, as no prior validated tools have been developed to specifically measure KAP regarding digital epidemiology (10). The availability of such a tool will facilitate future research by enabling systematic assessment and monitoring of digital epidemiology capacities in academic and professional settings. It also provides a foundation for designing tailored educational and policy interventions aimed at improving digital epidemiology adoption and application.
Despite its strengths, the study has limitations, including a geographically restricted sample limited to Iranian participants and the absence of criterion validity testing. Moreover, the questionnaire was developed in Persian, necessitating cross-cultural adaptation and validation for use in other languages and populations. Future research should aim to address these limitations and explore longitudinal validation to assess the stability of the instrument over time.
In summary, the validated KAP questionnaire represents a critical step forward in advancing digital epidemiology education and practice. Its use can enhance understanding of current awareness gaps, attitudes, and behaviors, thereby informing strategic initiatives to harness the full potential of digital technologies in epidemiological research and public health interventions (10).
