Abstract
Background:
Data collection tools used in literature to evaluate the degree to which study objectives are fulfilled should be standard to ensure the accuracy of data.Objectives:
The present study was performed to standardize and validate a questionnaire designed for investigating the concepts of a health belief model of the style and pattern of using disposable plastic containers.Methods:
A review of literature suggests few questionnaires are available on the style and pattern of using disposable containers. In the first step of the present study, the first draft of the questionnaire was designed and its reliability and concepts were evaluated using expert comments. In the second step, its understandability was evaluated in a pilot study on 30 subjects. In the third and fourth steps, the questionnaire reliability was respectively examined using the test-retest and lot quality assurance sampling (LQAS) through calculating a Cronbach’s alpha in STATA.Results:
The questionnaire subscales included demographic information (8 items), perceived sensitivity (5 items), perceived severity (7 items), practical guidance (3 items), perceived benefits (9 items), perceived barriers (7 items), perceived self-efficiency (8 items), perceived pleasure (6 items) and interpersonal norms (4 items). The questionnaire was distributed among 20 individuals, and they were asked to examine its understandability. At least 80% of the items were acceptable as per quality assurance standards. After revising the questionnaire as required, it was presented to thirty samples. They were also asked to complete the questionnaire again within 15 days, and the results were evaluated for quality assurance. The items were then modified according to the degree to which the objectives were realized. Thirty participants ultimately completed the questionnaire, and the results were evaluated for quality assurance.Conclusions:
In a large body of literature, developing standard assessment tools has been considered the main step toward scientific research; nevertheless, standardizing a questionnaire in a way that it is made sensitive can be time-consuming. The present results confirmed the validity and reliability of the designed questionnaire for evaluating the health belief model of patterns of using disposable plastic containers.Keywords
Health Belief Model Disposable Containers Reliability Psychometric Assessment
1. Background
The changes in lifestyle caused by population growth and industrial development have increased the use of disposable containers. The Research Center of the Islamic Consultative Assembly of Iran has reported the disposal of 2.1 million tons of plastic per year, which could have entered the solid waste recycling process (1, 2). Today, low- and high-density polyethylene containers are widely used in Iran for keeping foods and beverages, as has been the case over the past 30 years. As the main material in these containers, polystyrene has been frequently reported as a harmful and cancerous compound. Packaging hot, i.e. over 65ºC, and greasy foods was therefore forbidden by the Iranian Ministry of Health and Medical Education (3-5). In Iran, disposable plastic containers are extensively used on different occasions, e.g. celebrations and gatherings, for keeping hot and greasy foods and beverages regardless of their negative effects. The transmission of bisphenol and nonylphenol from these containers to food and drinkable items is therefore highly probable. The use of disposable containers produced from vegetables such as corn starch, rice, potato and wheat is currently growing in many countries. These containers are degraded within 3 - 6 months, and livestock can be fed with their debris. Moreover, phenols are rarely transmitted from these containers to the food chain (6-8). Given the significant advances made in different social, educational and research fields, the application and assessment of questionnaires and checklists have attracted the attention of many researchers (9). Validity and reliability assessments have been proposed to ensure the accuracy of the capability measurements made in humanity and behavioral studies (10). The tools used to investigate the style and pattern of using disposable plastic containers should be adequately reliable and valid. In general, reliability and validity are respectively considered the indicators of random errors and systematic or measurement errors. Validity determines the degree to which practical measurements conform with the objectives set (9-12).
Nourbakhsh et.al investigated the knowledge and behavior assessment about the use of disposable plastic containers amongst medical sciences students in northeastern Iran, and found only 5% of the respondents to be familiar with safety codes on the bottom of disposable plastic containers. They observed no statistically-significant differences in the mean score of knowledge by gender, marital status and education level (2). Siavoshi and Sharifi, examined the role of knowledge of the environment in promoting disposable tableware, and found this knowledge as well as the use of religious messages to increase willingness to buy these containers, especially in religious ceremonies (13). Khazraei studied the role of training on the use of disposable tableware in reducing environmental pollution, and found that training on the environment could involve significantly effective variables such as knowledge of the environment, attitude towards the environment and attention to health (14).
Content validity and face validity should be measured for determining validity. In this regard, the model developed by Lawshe for content validity assessments involved a panel comprising experts in different fields, who were asked to complete the questionnaires (10). Face validity is generally a subjective judgment about the structure of the designed tool, and examines the conformity of the tool appearance with the research objectives. In other words, face validity is the degree to which panelists’ understanding of the questionnaire conforms with the authors’. Research suggests that unreliability of a questionnaire is significantly associated with its invalidity. Such a questionnaire cannot therefore yield accurate data. Overall, reliability and validity should not be examined independently, as improving reliability enhances validity and vice versa (15-19).
2. Objectives
The present study was therefore conducted to design an instrument for investigating the style and pattern of using disposable plastic containers, and perform a psychometric assessment on this tool.
3. Methods
The present descriptive cross-sectional study was conducted to propose a tool for investigating the style and pattern of using disposable containers, and conduct a psychometric assessment on this tool. Given the need for a specific assessment tool, a researcher-made questionnaire was used on the style and pattern of using disposable containers. Demographic variables used in reliability assessments included age, gender, education level, occupational status, marital status, place of residence, residence status and household economic status. In the first step, the designed questionnaire was presented to twenty health and hygiene specialists, and they were asked to assess its face validity and content validity index (CVI). The first revision of the questionnaire was performed after collecting the comments of these experts. In the second step, this questionnaire was presented to twenty non-randomized individuals, and they were asked to assess its face validity and understandability. Conformity of the experts’ understanding of the questionnaire with the research objectives was determined as Yes/No, the results were stored in a computer, and LQAS was applied. The tool items were then selected, classified and evaluated on a five-point Likert scale and based on the research objectives and expert opinions. Content validity ratio (CVR) and CVI were also measured for determining content validity. The CVR of each item was assessed using the following formula (12):
in which ‘ne’ represents the number of individuals in the group emphasizing the necessity of the mentioned item, and ‘n’ the total number of study experts. Content validity was then calculated with the following formula (12):
Furthermore, the face validity of the tool as an indicator of item importance was calculated using the following formula:
in which ‘Importance score’ represents the score of the item importance, ‘frequency’ the number by which the score is repeated, ‘N’ the total number of experts, and ‘importance’ the number of each item (1-4, 14).
Moreover, the relatedness standard was separately evaluated on a four-point Likert scale including 1: unrelated, 2: relatively related, 3: related and 4: completely related. The face validity index was calculated for each item by dividing the number of experts agreeing (scores 3 and 4) by the total number of experts. Given that the minimum acceptable score for face validity was 1.5, the items with scores below 1.5 were eliminated (15).
In the third step, the questionnaire reliability was examined using test-retest assessments with 15-day intervals via administering the questionnaire on thirty non-randomized individuals. Cronbach's alpha coefficient was determined in STATA to calculate LQAS. In the fourth step, the questionnaire reliability was determined based on a test-retest method by administering the questionnaire on thirty 30 samples, re-administering the questionnaire within 15 days, and measuring LQAS and Cronbach’s alpha in STATA. The questionnaire was ultimately confirmed to be appropriate for extensive applications. The data were presented by measuring descriptive statistics.
4. Results and Discussion
According to Table 1, the subscales of the designed questionnaire with a total of 57 items included demographic information (8 items), perceived sensitivity (5 items), perceived severity (7 items), practical guidance (3 items), perceived benefits (9 items), perceived barriers (7 items), perceived self-efficiency (8 items), perceived pleasure (6 items) and interpersonal norms (4 items).
Results of Field Investigation for Face Validity Assessment and Questionnaire Readabilitya
| Domains of Questions of the Questionnaire | The Number of Questions That Enjoy Maximum Acceptable Face Validity | Total | |
|---|---|---|---|
| Maximum Acceptable Error 5% | Maximum Acceptable Error 15% | ||
| Demographic | 8 (100) | 8 (100) | 8 (100) |
| perceived sensitivity | 3 (60) | 2 (40) | 5 (100) |
| Perceived severity | 4 (57.1) | 5 (71.4) | 7 (100) |
| Practical guidance | 2 (66.7) | 1 (33.3) | 3 (100) |
| Perceived benefits | 5 (55.5) | 4 (44.4) | 9 (100) |
| Perceived stoppages | 2 (28.6) | 6 (85.7) | 7 (100) |
| Perceived self-efficiency | 6 (75) | 2 (25) | 8 (100) |
| Perceived related pleasure | 5 (83.3) | 2 (33.3) | 6 (100) |
| Interpersonal norms | 3 (75) | 2 (50) | 4 (100) |
| Total | 38 (66.6) | 32 (56.1) | 57 (100) |
In the first stage, expert comments on merging or eliminating some items and adding some other items were implemented. After making the required revisions, the subscales of the designed questionnaire included perceived sensitivity (5 items), perceived severity (7 items), practical guidance (3 items), perceived benefits (9 items), perceived barriers (7 items), perceived self-efficiency (7 items), perceived pleasure (4 items) and interpersonal norms (4 items).
In the second step, two levels were considered as per the quality assurance standard for the maximum acceptable error, i.e. 5% (63.3% of items) and 15% (53.3% of items) (Table 1). In the third step, the highest internal reliability and repeatability were reported for the demographic (100% reliability), self-efficiency (Cronbach’s alpha = 0.757) and perceived benefits (Cronbach’s alpha=0.725) subscales (Table 2). On the other hand, the lowest reliability was respectively associated with practical guidance (Cronbach’s alpha = 0.384) and perceived severity (Cronbach’s alpha = 0.422).
The Results of the Third Stage Field Survey to Measure Reliability and Repeatabilitya
| Domains of Questions of the Questionnaire | Alpha Cronbach’s Statistic | Repeatability | ||
|---|---|---|---|---|
| Excellent (Less than 5% Error) | Acceptable (Between 5% and 15% Error) | More than 15% Error | ||
| Demographic | b | 8 (100) | 0 | 0 |
| Perceived sensitivity | 0.565 | 2 (40) | 2 (40) | 1 (20) |
| Perceived severity | 0.422 | 2 (28.6) | 3 (42.8) | 2 (28.6) |
| Practical guidance | 0.384 | 2 (66.7) | 0 | 1 (33.3) |
| Perceived benefits | 0.725 | 6 (66.7) | 1 (11.1) | 2 (22.2) |
| Perceived stoppages | 0.615 | 2 (28.6) | 4 (57.1) | 1 (14.3) |
| Perceived self-efficiency | 0.757 | 4 (50) | 4 (50) | 0 |
| Perceived related pleasure | 0.451 | 3 (50) | 1 (16.7) | 2 (33.3) |
| Interpersonal norms | 0.572 | 1 (25) | 2 (50) | 1 (25) |
| Total | 0.538 | 30 (52.6) | 17 (29.9) | 10 (17.5) |
The fourth stage of the study was completed after making the final revisions involving the investigation of the questionnaire. The results suggested an increase in the internal reliability of the whole questionnaire based on a Cronbach’s alpha of 0.538 - 0.714 and in its repeatability based on the repeatability of each subscale of the questionnaire (Table 3).
The Results of the Fourth Stage Field Survey to Measure Reliability and Repeatabilitya
| Domains of Questions of the Questionnaire | Alpha Cronbach’s Statistic | Repeatability | ||
|---|---|---|---|---|
| Excellent (Less than 5% Error) | Acceptable (Between 5% and 15% Error) | More than 15% Error | ||
| Demographic | b | 8 (100) | 0 | 0 |
| Perceived sensitivity | 0.766 | 4 (80) | 1 (20) | 0 |
| Perceived severity | 0.726 | 4 (57.1) | 2 (28.6) | 1 (14.3) |
| Practical guidance | 0.791 | 2 (66.7) | 1 (33.3) | 0 |
| Perceived benefits | 0.768 | 6 (66.7) | 3 (33.3) | 0 |
| Perceived stoppages | 0.697 | 4 (57.1) | 3 (42.9) | 0 |
| Perceived self-efficiency | 0.732 | 4 (50) | 4 (50) | 0 |
| Perceived related pleasure | 0.755 | 4 (66.7) | 2 (33.3) | 0 |
| Interpersonal norms | 0.727 | 2 (50) | 1 (25) | 1 (25) |
| Total | 0.714 | 38 (66.7) | 17 (29.9) | 2 (3.5) |
The present study was performed to determine the reliability, validity, face repeatability and readability of the designed questionnaire. The initial tool was first presented to twenty experts, who investigated different scientific contents. Given the limited number of study domains, a limited number of experts were required to be asked for providing comments. Opinions of more experts and scientists would have been therefore required to be collected if a higher number of aspects were to be considered in the samples. The questionnaire repeatability and reliability were confirmed by calculating a Cronbach’s alpha of over 0.7.
4.1. Conclusions
In addition to health experts, a panel of experts in different scientific fields, including environmental health, health services management and vital statistics, were asked to share their views in the present study. Maximum variation was observed by including the comments and viewpoints of different experts in the design of the tool. Repeating the tests and re-administering the questionnaire for developing a tool with the maximum efficiency were other strengths of the present study. Despite being time-consuming, re-administration enabled the researcher to ensure the accuracy of both the data obtained and the conclusion.
Acknowledgements
References
-
1.
Kazemi A, Younesi H, Bahramifar N. [Migration of bisphenol A and nonylphenol from mineral water bottles and disposable plastic containers into water at different temperatures]. Iran J Health Environ. 2014;6(4):515-22. Persian.
-
2.
Nourbakhsh S, Aghaee MA, Salmani ER, Naderi M, Zangi R, Feizi R. Knowledge and behavior assessment about the use of disposable plastic containers amongst medical sciences students in Northeastern Iran in 2016. Iran J Health Saf Environ. 2017;4(3):804-11.
-
3.
Tawfik MS, Huyghebaert A. Polystyrene cups and containers: Styrene migration. Food Addit Contam. 1998;15(5):592-9. [PubMed ID: 9829045]. https://doi.org/10.1080/02652039809374686.
-
4.
Thompson RC, Moore CJ, vom Saal FS, Swan SH. Plastics, the environment and human health: Current consensus and future trends. Philos Trans R Soc Lond B Biol Sci. 2009;364(1526):2153-66. [PubMed ID: 19528062]. [PubMed Central ID: PMC2873021]. https://doi.org/10.1098/rstb.2009.0053.
-
5.
Cano M. Primary materials. Jane Austen and Performance. 2017. https://doi.org/10.1007/978-3-319-43988-4.
-
6.
Khaksar MR, Ghazi-Khansari M. Determination of migration monomer styrene from GPPS (general purpose polystyrene) and HIPS (high impact polystyrene) cups to hot drinks. Toxicol Mech Methods. 2009;19(3):257-61. [PubMed ID: 19750020]. [PubMed Central ID: PMC2739648]. https://doi.org/10.1080/15376510802510299.
-
7.
Cook DA, Beckman TJ. Current concepts in validity and reliability for psychometric instruments: Theory and application. Am J Med. 2006;119(2):166 e7-16. [PubMed ID: 16443422]. https://doi.org/10.1016/j.amjmed.2005.10.036.
-
8.
Merriam S. What Can You Tell From An N of 1?: Issues of validity and reliability in qualitative research. PAACE J Lifelong Learn. 1995:51-60.
-
9.
Golafshani N. Understanding reliability and validity in qualitative research. Qual Rep. 2003;8(4):597-606.
-
10.
Lawshe CH. A quantitative approach to content validity. Pers Psychol. 1975;28(4):563-75. https://doi.org/10.1111/j.1744-6570.1975.tb01393.x.
-
11.
Rashidi A, Lotphiyar M, Seifrabie MA, Mahini F, Alikhani MY. Microbial contamination and antibiotic resistance of bacterial isolates from disposable food containers in Hamadan. Pajouhan Sci J. 2015;13(4):27-33.
-
12.
Drost EA. Validity and reliability in social science research. Educ Res Perspect. 2011;38(1):105.
-
13.
Siavoshi M, Sharifi Y. Investigating the use of environmental knowledge in the propagation of vegetable containers, on the tendency to buy these products. The First International Conference on Innovation, Development and Business: Kian Pajohan Scientific Institute. 2015.
-
14.
Khazraei L. Effect of training on the use of vegetable dishes on reducing environmental contamination. Fourth Conference on Environment, Energy and Biological Defense: Higher Education Institute of Mehr Arvand - Extension Group for Environmental Lovers. 2015.
-
15.
Hyrkäs K, Appelqvist-Schmidlechner K, Oksa L. Validating an instrument for clinical supervision using an expert panel. Int J Nurs Stud. 2003;40(6):619-25. https://doi.org/10.1016/s0020-7489(03)00036-1.
-
16.
Mohammad Beigi A, Tabatabaei H, Zeighami B, Mohammad Salehi N. Determination of diabetes risk factors during pregnancy among women reside in Shiraz. Iran J Diabetes Metab. 2007;7(1):77-84. https://doi.org/10.4103/0973-3930.34751.
-
17.
Wells CS, Wollack JA. An instructor’s guide to understanding test reliability. Testing and Evaluation Services. University of Wisconsin; 2003.
-
18.
Guttman L. A basis for analyzing test-retest reliability. Psychometrika. 1945;10:255-82. [PubMed ID: 21007983].
-
19.
Bland JM, Altman DG. Statistics notes: Cronbach's alpha. BMJ. 1997;314(7080):572. [PubMed ID: 9055718]. [PubMed Central ID: PMC2126061]. https://doi.org/10.1136/bmj.314.7080.572.