Abstract
Background:
Addiction to virtual social networks (VSNs), especially among students, has become a crisis during the recent years.Objectives:
This study aimed to assess the validity and reliability of a questionnaire on VSN addiction among Iranian university students.Patients and Methods:
The initial questionnaire was designed based on extensive literature review and consulting with experts. To measure the item and content validity indexes (I-CVI and S-CVI) and to measure the content validity ratio (CVR), a panel of 24 experts reviewed the questionnaire. To measure the test-retest reliability, the questionnaire was administered on 30 students within the interval of 14 - 21 days and the intra-class correlation coefficient (ICC) was calculated. Cronbach’s alpha and the corrected item-total correlation were calculated to measure internal consistency. All analysis was done using SPSS 20 software.Results:
All items had satisfactory CVR and I-CVI. The S-CVI was 0.98. The value of Cronbach’s alpha was 0.88. The corrected item-total correlation for all items, except one, was in acceptable range. This item was removed from the final questionnaire. The test-retest reliability of the questionnaire was almost perfect (ICC = 0.9).Conclusions:
The current study provides a valid and reliable questionnaire to measure VSN addiction among university students. The designed instrument could be used in addiction evaluation studies.Keywords
Behavior Addictive Social Networking Validation Study Reliability Student Psychometrics
1. Background
Virtual social networks (VSNs) have gained popularity over the past decades and millions of users are currently using them (1, 2). These networks are so popular that life seems to be difficult without them. These VSNs have turned into a new way of communication, building relationships, finding new friends, entertainment and communication among people all over the world (3, 4), and have changed the way and quality of interpersonal communications significantly (5).
The attractiveness of virtual social networks has made people spend a lot of time online (6). There are a variety of terms for describing the overuse of the Internet, among them “Internet addiction” is more commonly used. This term is used for excessive and harmful use of the internet (7). Addiction to VSNs is a subset of Internet addiction which has become especially problematic within the recent years (6). It is shown that Internet addiction fulfils all the main components of the addiction model proposed by Griffiths in 2005 (8). These components are salience, tolerance, mood modifications, withdrawal, conflict and relapse. In fact, people who are addicted to VSNs spend a lot of time thinking of them and try to find opportunities to use these networks (salience). They often spend more time on these networks than they have planned, and try to increase the duration of their use to achieve enjoyment (tolerance). In addition, they use these networks to escape from the loneliness and depression (mood modifications) and if they are banned from using these networks, they get stressed, restless and irritable (withdrawal). They do not pay attention to the advice of others to diminish the time that they spend on these networks, and even when they decide to reduce the use of VSNs, they would not succeed (relapse). Other activities including their study, sports, and being with their families will be of less importance to these people, and it affects their quality of sleep, health, and communication (conflict) (8).
University students are a group at higher risk of Internet addiction. Around 85% of students use VSNs as the main root of communication (9). Establishing the communication regardless of distance, inserting and reading the university announcements, saving time and costs are the benefits of using these networks among students. However, spending too much time on VSNs may have negative effects on the academic performance of students (10, 11). In a study conducted in Birjand, Iran, the prevalence of mild and severe Internet addiction among high school students was reported to be 20.5% and 5.8% respectively (12). Another study in Iran showed that around 83.61% of university students suffer from mild to moderate Internet addiction (13).
Most studies conducted on Internet addiction among Iranian students were conducted before the emergence and widespread use of new VSNs such as WhatsApp and Telegram. Furthermore, they only focused on the Internet addiction while the problem of the recent years in not the Internet addiction but addiction to VSNs. In addition, most studies used researcher made questionnaires that their validity and reliability haven’t been evaluated properly.
2. Objectives
As a result, we conducted the present study to design and validate a questionnaire to measure VSN addiction among university students.
3. Patients and Methods
3.1. Instrument Development
After extensive literature review, we found variety of instrument used for identifying Internet addiction. These instruments were the Young Internet Addiction questionnaire (14) and the questionnaires and scales used in studies conducted by Mahmood and Farooq (15), Matar Boumosleh and Jaalouk (16), Savari et al. (17), Obaidi et al. (18), Liuokai et al. (19) and Bodroza and Jovanovic (20). After precise reading of these instruments and discussion with experts, we designed a questionnaire which could capture most dimensions of VSN addiction. The Questionnaire Consisted of Three Parts: The forepart included 3 demographic questions on gender (female, male), level of education (undergraduate, postgraduate, doctoral, PhD) and place of residence (dormitory, non-dormitory). The second part contained 3 questions on (1) the most common purpose of using VSNs; (2) the types of VSNs commonly used; and (3) the average time that students spend daily on using them. In the final part, we asked students 15 questions on various aspects of VSN addiction. The answering scheme contained two types of five point Likert scale including “never, slightly, moderate, high, and very high” for questions 1 - 5 and “never, rarely, sometimes, most often, and always” for questions 6 - 15.
3.2. Evaluating the Face and Content Validity
As most researchers believe that the face validity is a component of content validity and it is not an active measure of validity (21), we measured both face and content validity in the same manner using qualitative and quantitative approaches. In the qualitative phase, a team of 7 experts including two psychiatrists, two psychologists and three epidemiologists assessed the first version of the designed questionnaire through a face-to-face interview. Their suggestions for revision were addressed and the questions were modified according to their comments. Then the content validity of the modified version was evaluated quantitatively by calculating the content validity index (CVI) and content validity ratio (CVR).
3.3. Content Validity Ratio
The CVR was calculated for each question based on the formula developed by Lawsche. A panel of 24 experts including psychologists, psychiatrists, epidemiologists and health education and promotion specialists, were asked to review and rate the appropriateness of each question using a three point scale: “essential,” “useful but not essential,” or “not necessary.” Calculated CVRs were compared to the value of 0.41, the minimum required value for 24 raters based on Lawsche’s table (22, 23).
3.4. Content Validity Index
The content validity of the questionnaire was assessed using item and scale content validity index (I-CVI and S-CVI). The experts panel were asked to score the clarity, simplicity and relevance of each question using a four point Likert scale (ranging from not relevant/unclear to completely relevant/clear). To calculate the I-CVI for each item, the proportion of experts who gave a rating of either 3 or 4 to the total number of experts was computed (24, 25). To measure the S-CVI, the average of all I-CVIs was calculated (21, 26). Content validity indexes were considered to be acceptable when I-CVI and S-CVI were at least 0.78 and 0.90 respectively.
3.5. Evaluating the Reliability
To confirm the reliability of the questionnaire, two methods including the internal reliability and test-retest reliability were used. In order to measure the internal reliability and consistency of the questionnaire, Cronbach’s alpha and corrected item-total correlation were calculated. The minimum acceptable coefficient for Cronbach’s alpha was set at 0.7. To calculate the corrected item-total correlation, the correlation of each item with the total score of the questionnaire was assessed by calculation of the Pearson correlation coefficient. The items which had a coefficient of more than 0.3 were accepted (27). To measure the test-retest reliability, 20 students underwent two administrations of questionnaires with the interval of 14 - 21 days. Then, the intra-class correlation coefficients (ICC) were calculated. The coefficient of 0 - 0.2 was considered as slight, 0.21 - 04 as fair, 0.41 - 0.6 as moderate, 0.61-0.8 as substantial and > 0.8 as almost perfect reliability. All analysis were done using SPSS 20 software.
3.6. Ethics
The study protocol was reviewed and approved by the Ethics Committee of Kerman University of Medical Sciences (ethic code: IR.KMU.REC.1396.1806).
4. Results
The demographic characteristics of students who participated in the reliability study is presented in Table 1.
Demographic Characteristics of Students Recruited in Test-Retest Reliability Study of Questionnaire on Virtual Network Addiction Among University Students
| Number | |
|---|---|
| Gender | |
| Female | 10 |
| Male | 10 |
| Education | |
| Undergraduate | 8 |
| Postgraduate | 8 |
| Doctoral | 2 |
| PhD | 2 |
| Place of residence | |
| Dormitory | 10 |
| Non-dormitory | 10 |
The quantitative measures of validity and reliability are summarized in Table 2. All items had satisfactory CVR and I-CVI (Table 2). The S-CVI calculated for the whole questionnaire was 0.98. The value of Cronbach’ alpha was also satisfactory (0.88). The corrected item-total correlation for all items except the item number 3 was in acceptable range (Table 2). This item was removed from the final questionnaire. The test-retest reliability of the questionnaire was almost perfect (ICC = 0.9). The final questionnaire consisted of 3 demographic questions (gender, education and place of residence), 1 question on the most common purpose of using VSNs, 1 question on the type of VSNs used, 1 question on average time that someone spent daily on using VSNs and 14 questions about VSNs addiction.
Test-Retest and Content Validity Indicators of Questionnaire on virtual Network Addiction Among University Students, Kerman, Iran
| Row | Phrases | CVR | I-CVI Simplicity | I-CVI Clarity | I-CVI Relate | Corrected Item-Total Correlation |
|---|---|---|---|---|---|---|
| To what extend | ||||||
| 1 | Spending time on VSANs has harmed your academic education? | 0.91 | 1 | 1 | 1 | 0.66 |
| 2 | Do you prefer to go online than going out with your friends? | 0.91 | 1 | 1 | 1 | 0.5 |
| 3 | Do you prefer to go online than going out with your family? | 0.83 | 1 | 1 | 1 | 0.45 |
| 4 | You can predict the time that you go back to use VSNs again? | 0.75 | 0.87 | 0.91 | 1 | 0.11 |
| 5 | Spending time on VSNs has adversely affected your work? | 0.58 | 1 | 1 | 1 | 0.74 |
| 6 | If someone disturbs you while you are online on VSNs make you upset, angry or screaming? | 0.58 | 1 | 1 | 1 | 0.55 |
| 7 | You tell this sentence to others while you are using VSNs: “Just a few minutes; I’m coming now” | 0.83 | 1 | 1 | 1 | 0.67 |
| 8 | Have you tried to shorten the time you spend on VSNs, but you couldn’t do that? | 0.91 | 1 | 1 | 1 | 0.61 |
| 9 | Do you try to hide from others that you are online? | 0.75 | 1 | 1 | 1 | 0.44 |
| 10 | Have you ignored your family members because of being online? | 0.91 | 1 | 0.95 | 0.95 | 0.64 |
| 11 | Other people complained to you because of being online? | 0.75 | 1 | 1 | 1 | 0.65 |
| 12 | Do you lose your sleep because of being late at night on VSNs? | 0.83 | 0.95 | 0.95 | 1 | 0.64 |
| 13 | How much longer have you spent on VSNs than you planned? | 0.83 | 1 | 0.95 | 0.87 | 0.54 |
| 14 | Have you checked VSNs before doing your priority tasks? | 1 | 1 | 1 | 1 | 0.59 |
| 15 | Reading a message on VSNs occupies your mind when you get out of it? | 0.91 | 1 | 1 | 1 | 0.49 |
5. Discussion
The purpose of present study was to develop a standard questionnaire for measurement of addiction to virtual social networks among students. The review of the previous studies showed a gap in measurement of addiction to VSNs. Therefore, the need for a specific instrument for measurement of this disorder made us develop and validated a questionnaire for this purpose. One of the strengths of the present study is that we asked a team of experts, including 24 different specialists in the field of addiction and psychology to review and validate the questionnaire. Furthermore, we recruited a representative sample of students from different fields of study and degrees for reliability of the study. Therefore, the instrument would be applicable in a wide range of settings. In the design stage, we tried to address most components of addiction suggested by Griffiths in 2005. These components are salience, mood modification, tolerance, withdrawal, conflict and relapse (8). The conflict is defined by Griffiths as the “conflict between the addict and those around them (interpersonal conflict) or from within the individual themselves (intrapsychic conflict) which are concerned with the particular activity” (8). As the conflict itself has a variety of aspects, 8 of our questions were assigned to this component. The questions number 2, 3, and 10 which ask about compromised relationship of individuals with their family and friends measure the personal conflicts while questions 9 and 11 measure the social conflicts that affect social life of individuals. In addition, questions 1 and 5 measure the educational and work conflict respectively.
Another component of addiction based on the Griffiths model is relapse and refers to “the tendency for repeated reversions to earlier patterns of the particular activity to recur and for even the most extreme patterns typical of the height of the addiction to be quickly restored after many years of abstinence or control” (8). We addressed this component by asking the question number 6: “have you tried to shorten the time you spend on VSNs, but you couldn’t do that?”. To measure the salience, which is defined as the situation “when the particular activity becomes the most important activity in the person’s life and dominates their thinking (preoccupations and cognitive distortions), feelings (cravings) and behavior (deterioration of socialized behavior” (8) we asked the questions number 12: “Do you lose your sleep because of being late at night on VSNs?”, number 14: “Have you checked VSNs before doing your priority tasks?” and number 15: “Reading a message on VSNs occupies your mind when you get out of it?”. The question number 6 asks about the withdrawal component of the addiction model which is defined as “unpleasant feeling states and/or physical effects which occur when the particular activity is discontinued or suddenly reduced” (8). Finally, we measured the tolerance, “the process whereby increasing amounts of the particular activity are required to achieve the former effects” (8), by asking the question number 13: “How much longer have you spent on VSNs than you planned?”.
In addition to the comprehensiveness of the designed questionnaire in measuring various components of addiction, the high value of I-CVI and S-CVI confirms the high validity of the instrument. Furthermore, the high value of Cronbach’s alpha coefficient and ICC and acceptable values of corrected item-total correlation indicate the satisfactory reliability of the questionnaire.
We would like to acknowledge the limitation of our study. We just measured the face and content validity of the questionnaire which are the most important aspects of questionnaire validation. Other aspects of validity including criterion, convergence and predictive validity were not addressed in this study.
5.1. Conclusions
The designed questionnaire provides a standardized, valid, and reliable instrument to measure the extent of virtual social network addiction among Iranian university students. The instrument could be applicable in routine evaluation of students and designing educational interventions.
Acknowledgements
References
-
1.
Pelling EL, White KM. The theory of planned behavior applied to young people's use of social networking Web sites. Cyberpsychol Behav. 2009;12(6):755-9. [PubMed ID: 19788377]. https://doi.org/10.1089/cpb.2009.0109.
-
2.
Wilson K, Fornasier S, White KM. Psychological predictors of young adults' use of social networking sites. Cyberpsychol Behav Soc Netw. 2010;13(2):173-7. [PubMed ID: 20528274]. https://doi.org/10.1089/cyber.2009.0094.
-
3.
Raacke J, Bonds-Raacke J. MySpace and Facebook: applying the uses and gratifications theory to exploring friend-networking sites. Cyberpsychol Behav. 2008;11(2):169-74. [PubMed ID: 18422409]. https://doi.org/10.1089/cpb.2007.0056.
-
4.
Hanany Abd Rahman S. Can’t Live without my FB, LoL: The influence of Social Networking Sites on the communication skills of TESL Students. Proced Soc Behav Sci. 2014;134:213-9. https://doi.org/10.1016/j.sbspro.2014.04.241.
-
5.
Stieger S, Burger C, Bohn M, Voracek M. Who commits virtual identity suicide? Differences in privacy concerns, Internet addiction, and personality between Facebook users and quitters. Cyberpsychol Behav Soc Netw. 2013;16(9):629-34. [PubMed ID: 23374170]. https://doi.org/10.1089/cyber.2012.0323.
-
6.
Kuss DJ, Griffiths MD. Online social networking and addiction--a review of the psychological literature. Int J Environ Res Public Health. 2011;8(9):3528-52. [PubMed ID: 22016701]. [PubMed Central ID: PMC3194102]. https://doi.org/10.3390/ijerph8093528.
-
7.
Spada MM. An overview of problematic internet use. Addict Behav. 2014;39(1):3-6. [PubMed ID: 24126206]. https://doi.org/10.1016/j.addbeh.2013.09.007.
-
8.
Griffiths MD. A ‘components’ model of addiction within a biopsychosocial framework. J Subst Use. 2009;10(4):191-7. https://doi.org/10.1080/14659890500114359.
-
9.
Heiberger G, Harper R. Have you facebooked Astin lately? Using technology to increase student involvement. New Dir Stud Serv. 2008;2008(124):19-35. https://doi.org/10.1002/ss.293.
-
10.
Seder JP, Oishi S. Ethnic/racial homogeneity in college students’ Facebook friendship networks and subjective well-being. J Res Pers. 2009;43(3):438-43. https://doi.org/10.1016/j.jrp.2009.01.009.
-
11.
Thompson LA, Dawson K, Ferdig R, Black EW, Boyer J, Coutts J, et al. The intersection of online social networking with medical professionalism. J Gen Intern Med. 2008;23(7):954-7. [PubMed ID: 18612723]. [PubMed Central ID: PMC2517936]. https://doi.org/10.1007/s11606-008-0538-8.
-
12.
Turi A, Miri M, Beheshti D, Yari E, Khodabakhshi H, Anani Sarab G. Prevalence of Internet addiction and its relationship with anxiety, stress, and depression in intermediate students in Birjand City in 2014. J Birjand Univ Med Sci. 2015;22(1):67-75.
-
13.
Doosti Irani A, Bagheri Amiri F, Khajehkazemi R, Mostafavi E. Prevalence of internet addiction among students and graduates of epidemiology, clinical sciences, and basic sciences in Iran: A cross-sectional study. Iran J Epidemiol. 2017;13(1):14-21.
-
14.
Faraci P, Craparo G, Messina R, Severino S. Internet Addiction Test (IAT): Which is the best factorial solution? J Med Internet Res. 2013;15(10). e225. [PubMed ID: 24184961]. [PubMed Central ID: PMC3806548]. https://doi.org/10.2196/jmir.2935.
-
15.
Mahmood S, Farooq U. Facebook addiction: A study of big-five factors and academic performance amongst students of IUB. Global J Manage Bus Res. 2014.
-
16.
Matar Boumosleh J, Jaalouk D. Depression, anxiety, and smartphone addiction in university students- A cross sectional study. PLoS One. 2017;12(8). e0182239. [PubMed ID: 28777828]. [PubMed Central ID: PMC5544206]. https://doi.org/10.1371/journal.pone.0182239.
-
17.
Savari K. Build and accelerate mobile addiction inventory. Educ Measur. 2014;4(15):126-42.
-
18.
Obaidi N, Kiani MR/, RostamiNejad MA. [Validation of questionnaire of psycho-social aspects of phone based social networks' use]. Hum Inf Interact. 2016;3. Persian.
-
19.
Shi Y, Luo YLL, Yang Z, Liu Y, Cai H. The development and validation of the Social Network Sites (SNSs) usage questionnaire. 8531. Germany: Springer; 2014. p. 113-24. https://doi.org/10.1007/978-3-319-07632-4_11.
-
20.
Bodroza B, Jovanovic T. Validation of the new scale for measuring behaviors of Facebook users: Psycho-Social Aspects of Facebook Use (PSAFU). Computers Hum Behav. 2016;54:425-35. https://doi.org/10.1016/j.chb.2015.07.032.
-
21.
Bolarinwa OA. Principles and methods of validity and reliability testing of questionnaires used in social and health science researches. Niger Postgrad Med J. 2015;22(4):195-201. [PubMed ID: 26776330]. https://doi.org/10.4103/1117-1936.173959.
-
22.
Jafari MJ, Yazdi FDF, Mehrabi Y, Rakhshanderou S, Saremi M. Development and validation of a new scale for prediction of low back pain occurrence among nurses. EXCLI J. 2019;18:277.
-
23.
Wilson FR, Pan W, Schumsky DA. Recalculation of the critical values for Lawshe’s content validity ratio. Measur Eval Couns Dev. 2017;45(3):197-210. https://doi.org/10.1177/0748175612440286.
-
24.
Joshi A, Kale S, Chandel S, Pal D. Likert scale: Explored and explained. Br J Appl Sci Tech. 2015;7(4):396-403. https://doi.org/10.9734/bjast/2015/14975.
-
25.
Asun RA, Rdz-Navarro K, Alvarado JM. Developing multidimensional Likert scales using item factor analysis. Soc Methods Res. 2015;45(1):109-33. https://doi.org/10.1177/0049124114566716.
-
26.
Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health. 2007;30(4):459-67. [PubMed ID: 17654487]. https://doi.org/10.1002/nur.20199.
-
27.
Mukaka MM. A guide to appropriate use of correlation coefficient in medical research. Malawi Med J. 2012;69(71):69-71.