Assessing the Quality of Online Learning During the COVID-19 Pandemic from the Perspective of Iranian Community Health Workers and Their Instructors

1. Background

Worldwide, e-learning has emerged as one of the most important applications of information and communication technology developments in recent decades. This type of education, known as education with no physical presence, is a new teaching method that uses information and communication technology (ICT). In e-learning, face-to-face training is completely simulated and can be divided into synchronous and asynchronous types. In the former, the exchange of information happens between trainees and instructors in real-time using tools such as live stream audio, video, and presentations. It allows students to participate in real-time discussions during class time and virtually attend classes with their instructor and classmates. Asynchronous learning allows students to use educational materials in texts, audio, and videos at any time. This type of education also provides the possibility of educational measurement, assessment, and evaluation through online exams and assignments (1). According to Castro, online learning has become popular due to its potential to provide more flexible access to educational content at any time and place. Its applicability, particularly in higher education, enables learners from different geographical areas to participate in an internet-based learning environment and learn at their own pace. Such an approach can also connect students of diverse backgrounds to obtain support throughout the learning process to construct knowledge and meaning (2). Recently, medical education has experienced a significant shift from traditional teaching styles to online, distance, or electronic learning, which applies electronic technology to support teaching strategies and engage learners in the learning process more effectively. This education is context-based (3).

In response to the COVID-19 pandemic, almost all countries, including Iran, put effort into preventing the spread of this infectious disease by applying different strategies such as social distancing, travel restrictions, lockdowns, and closure of educational institutions, etc. (4, 5). The Ministry of Education in all countries mandated all schools and academic institutions to implement distance learning by providing all educational curricula online (6). They used the advanced learning education system for educating and evaluating students to continue the education process during the pandemic (7).

During the pandemic, the use of different distance learning showed considerable progress and resulted in several benefits that traditional training programs failed to arrange. For example, having access to educational materials from anywhere at any time, asynchronous discussions with classmates and instructors, and immediate feedback on online exams and assignments are among the critical advantages of virtual education that facilitated the learning process during the pandemic (4). Several studies also affirmed the effectiveness of electronic learning systems (8-10). Despite the many advantages, these internet-based platforms still challenge academic institutions. Poor preparation of teachers for virtual teaching, improper access to information and technology infrastructures, being deprived of physical learning opportunities, low literacy of technological capabilities, unequal learning, and common challenges faced in the virtualization of some courses emphasized the need for the coordination of relevant managers and policymakers to plan coherently and comprehensibly for the future use of virtual learning (11). Aung and Khaing found that a lack of knowledge about information technology, poor infrastructure, and inadequate educational content caused severe challenges in implementing an effective e-learning system, particularly in developing countries (12). Kanwal and Rehman also revealed that computer self-efficacy is crucial for successfully adopting e-learning systems (13). Factors, including adequate information technology infrastructure, technical skills, and financial adequacy, were prerequisites for the proper implementation of e-learning projects (14, 15). In assessing students’ viewpoints about the quality of e-learning during the COVID-19 outbreak, Co et al. found a high level of psychological stress and declined academic performance among trainees (16). Recent studies in Iran have examined virtual education in medical universities during the COVID-19 epidemic. However, not many have focused on the experience of this education in Iranian community health worker (Behvarz) students and their instructors. The COVID-19 epidemic also affected the activities of these centers. According to the health regulations in Iran, the district Behvarz training center (DBTCs) of every University of Medical Sciences holds Behvarz pre-service training. Their study period is two years (17). Behvarz plays a vital role in community health. However, the quality of their training and the effectiveness of online learning have not been adequately evaluated, and COVID-19 offers the best opportunity to assess it.

2. Objectives

This study aimed to determine the quality of online learning from the perspective of learners and their instructors.

3. Methods

Using the census method, we conducted this cross-sectional study among 50 instructors and 150 trainees of community health workers (CHW) education programs at the Kermanshah University of Medical Sciences during 2020 - 2021. We included third-semester learners and teachers with over five years of experience in the study who had face-to-face training in the first semester and online training in the second semester. They lived in Kermanshah province and spread to 14 cities. The trainers were almost the bachelor in public health, environmental health, nursing, midwifery, and family health. Of these, 23 individuals had a master’s degree in community-based education. Following the COVID-19 pandemic and the suspension of face-to-face classes, educational materials were provided for learners virtually and online according to weekly scheduled programs. Instructors prepared the instructional design based on the Ministry of Health and Medical Education (MoHME) standards and supervised regularly. District Behvarz training center (DBTCs) tried to continue the practical training by sharing educational videos to provide practical lessons during this period. The educational videos were given to the students after the approval of the DBTCs. In addition, clinical placements were also considered for learners to facilitate gaining experience in the work environment and consequently positively affect their confidence and clinical and communication skills. We conducted this survey during the third trimester of a combined education program and used a 32-item questionnaire containing six dimensions of instructional design (6 items), learner-instructor interaction (4 items), learner-to-learner interaction (5 items), course content (6 items), individual learning opportunities (4 items), and course evaluation (7 items) to collect data. We developed the questions based on the criteria for evaluating the quality of e-learning in medical education mentioned in a questionnaire by Hakimzadeh and Afandideh (18). We asked participants to rate each of the items based on a five-point Likert scale from strongly agree to disagree strongly. The mean scores above 3.5, less than 3.5, and above four were relatively desirable, undesirable, and desirable, respectively. We assessed face validity with ten learners and five instructors and measured the content validity ratio (CVR) and content validity index by the Lawsche method.

A 20-member expert panel was formed with the specialties of medical education and virtual education and instructor of the Behvarz education program. CVR and CVI were 0.56 and 0.89, respectively. Therefore, content validity was appropriate. The reliability of each domain was measured with Cronbach’s alpha, which was greater than 0.7, and the mean of all domains was 0.79. Teachers and learners had a WhatsApp group with one researcher (manager of DBTCs). The researcher explained the questionnaire and then shared it with groups. We obtained informed consent from all participants, who could withdraw from the study whenever they wanted. We assured them about confidentiality during the research, the response rate was 100 percent for the two groups.

4. Results

We conducted this cross-sectional study among 50 trainers and 150 health Behvarz education program trainees.

There were 71 (47%) female and 79 (53%) male learners and 36 (72%) female, and 14 (28%) male instructors. The age range of instructor and learner was 20 - 65 and 16 - 28, respectively. The teaching experience of all trainers was over five years. Findings revealed that study participants were fairly satisfied with almost all educational aspects. The top four ranking dimensions were learner–instructor interaction, course content, learning opportunities, and educational design.

The greatest score in instructional design from the viewpoints of the trainer and learner is the adequacy of time required for learning and attention to the design of activities that help active learning, respectively. The lowest one in this domain from the viewpoints of the trainer and learner is attention to different learning styles of trainees and satisfaction from the learning environment, respectively. All scores of the learner in this domain are more significant than 3.5. However, the trainer's score is less than 3.5 in two items. The greatest score in the learner-instructor interaction from the viewpoints of the trainer and learner is giving prompt feedback to learners and easy access to instructors, respectively. The lowest one in this domain, from the viewpoints of the trainer and learner, is adequate to support and advise trainees during the learning process and encourage learners to participate in the learning process actively, respectively. All learner and trainer scores in this domain are more significant than 3.5. The greatest score in learner-to-learner interaction from the viewpoints of the trainer and learner is communicating with classmates through online communication platforms and ease of knowledge exchange between learners (chat, e-mail, etc.), respectively. The lowest one in this domain, from the viewpoints of the trainer and learner, is teamwork through computer-based communication facilities. One score from the learner's viewpoint and two from the learner's viewpoint are undesirable. The greatest score in course content from the viewpoints of the trainer and learner is the appropriateness of the type and size of the letters used in the educational content and referring to references used in educational content, respectively. The lowest one in this domain, from the viewpoints of the trainer and learner, is learners' progress at their own pace and the appropriateness of using a combination of text, sound, and images, respectively. One score from the trainer's viewpoint is undesirable, but all scores from the learner's viewpoint are desirable (> 3.5). The greatest score in individual learning opportunities from the viewpoints of the trainer and learner is providing an opportunity to increase information and control one's progress through an educational environment and the ability to learn at one's own pace, respectively. The lowest one in this domain, from the viewpoints of the trainer and learner, is using the training course according to one's learning strategies and ability to choose the place and time of learning, respectively. One score from the trainer's viewpoint is undesirable, but all scores from the learner's viewpoint are desirable. The greatest score in course evaluation from the viewpoints of the trainer and learner is that course assignments are based on the general, specific objectives of the training course and questions are answered on time, and assignments are appropriately reviewed, respectively. The lowest one in this domain, from the viewpoints of the trainer and learner, is the alignment of lesson exercises with the educational content and the provision of opportunities for learners to express what they have learned, respectively. Tree score from the viewpoints of the trainer is undesirable, but all scores from the learner's viewpoints are desirable Table 1.

The most significant and lowest score from the viewpoints of trainers and learners is learner- the instructor interaction dimension and learner-to-learner interaction, respectively (Table 2). Although one dimension is only undesirable from the learner’s viewpoint, two are undesirable from the trainer’s viewpoint. Learners’ scores are higher for the quality of virtual education.

5. Discussion

There was no difference between the two groups regarding the highest and lowest score for virtual learning quality. However, there were differences in each the details of each dimension. The score of online learning quality assessed by the learner and trainer is a relatively acceptable level except for learner-instructor interaction, which achieved the most excellent satisfaction among study participants. In contrast with our research findings, Iravani et al. found students' negative attitudes toward the e-learning approach during the COVID-19 pandemic, particularly in learner-instructor interaction and course evaluation (19). In another study, the influencing factors on e-learning quality were classified into four categories of culture; infrastructure (appropriate content, both the trainees' and trainers' capability to use electronic teaching platforms); process (teaching, evaluation, monitoring, support, and learner-instructor interaction); and output (planning for improvement). This study estimated the overall condition of e-learning at a moderate level (20). In a study conducted at the Kermanshah University of Medical Sciences during the COVID-19 pandemic, students evaluated their instructors at a moderate level and recommended teacher empowerment to develop their competencies in effective teaching methods and communication skills (20). By surveying similar studies in different countries, it was tried to provide a general overview of the state of the quality of virtual education in different environmental contexts. For example, in a study conducted at a private medical and dentistry school in Pakistan, students' viewpoints regarding the quality of e-learning during the COVID-19 pandemic were evaluated. Study findings revealed a moderate satisfaction level with the teaching approach's flexibility and efficiency. Students were pleased with easy access to educational materials and believed virtual education was more reliable and could increase their interest. Considering that the under-study university was for-profit, the quality of students' e-learning experience was more likely to be assessed at a higher level due to a more robust socio-economic background and relatively adequate infrastructure for virtual learning (21). These studies highlight the influential role of educational context and environmental factors in promoting e-learning (22). Developed countries face fewer problems implementing virtual education due to the desired speed and quality of internet connection (23). In addition, in these countries, justice in education has better conditions, and students have adequate access to information technology infrastructures and the internet to benefit from educational opportunities (21). In a similar study by Elzainy et al., an advanced online education was promoted to be used in the problem-based learning (PBL) approach during the COVID-19 pandemic, which proved more effective than face-to-face teaching classes (24). As a result of the review of existing literature, it has been concluded that establishing reliable and valid educational performance measurements, facilitating learner-instructor interaction, applying effective instructional methods, and ensuring that trainers and trainees are properly trained to utilize electronic resources are the best ways to improve the quality of virtual education. Therefore, objective standards are needed to ensure the quality of e-learning (25).

Several studies have investigated the challenges of e-learning in Iranian universities. Lack of effective communication between instructors and trainees; technical and infrastructural problems; divergence between the number of educational courses and the volume of contents; dissatisfaction with the quality of contents, online exams, and assignments; delayed feedback given to learners; lack of simulation of teaching content with face-to-face classes; and accumulation of a considerable volume of teaching content around the exam time were among the potential causes of dissatisfaction with the virtual teaching approach (21-27). Therefore, researchers in medical education proposed several strategies to resolve the issues. They emphasized that having information about the lesson objectives and evaluation methods at the beginning of the course and having a logical sequence in teaching and curriculum content aligned with learning objectives can lead to overall quality improvement for the training process (23). In a study by Maatuk et al. entitled "the COVID-19 pandemic and e-learning: Challenges and opportunities from the perspective of students and instructors", trainers believed that e-learning could develop students' technological skills. However, it might cause some challenges, particularly regarding inappropriate electronic infrastructure and lack of financial support. In this study, students declared their positive attitude toward the effects of e-learning and agreed on its potential impact on improved academic standards. Despite mentioned advantages, such as reduced costs, flexibility, and innovative learning approaches, they believed that one of the significant downsides of e-learning was imposed pressure on learning activities. This study revealed that the country's low-quality internet services were a significant challenge that required significant financial resources (28). Thus, it is essential to consider many recommendations from similar studies to overcome existing challenges. The provision of internet services to students and faculty members, establishing up-to-date electronic libraries with adequate devices, constant attention to information technology infrastructure, and periodic maintenance of computers and supporting equipment are among the necessary programs which should be taken into action to provide e-learning services properly (28). Studies in other countries have found this helpful training for community health workers. Among the studies conducted in Rwanda, virtual training for community health workers with a lower literacy level has increased the speed of teaching and learning them. They recognized this training as a continuous, cheap, and easy-to-use model for community health workers (29).

Compared to other studies, the current research depicts a more appropriate state of online learning from the viewpoints of trainees and instructors. This condition might be because of various reasons, such as a suitable student-teacher ratio in Behvarz education centers in Iran and more accessible learner-instructor interactions during the COVID-19 pandemic. A combined education was implemented in these centers, and the face-to-face training approach was still the dominant strategy to provide educational materials regarding practical sessions; higher satisfaction levels were observed regarding the quality of virtual education among the study participants. The shorter length of education and less complexity of scientific topics in the educational courses of Behvarz training and using more educational videos for the practical learning of trainees were, among other probable reasons, for a higher level of satisfaction.

Moving from traditional teaching methods to online methods can create opportunities for educational quality improvement. However, in order to resolve existing challenges, it is recommended to limit the size of online classes to facilitate prompt feedback to learners, consider an appropriate type of instructional design in line with study objectives, and mention new roles for instructors as site facilitators and non-traditional trainers in the distance learning process. Providing a continuous evaluation of students through standard quality indicators and promoting a well-established online environment can increase the appropriateness of study courses. A limitation of this study is that we cannot generalize the results to other centers. In addition, different participants' previous experiences with online education can affect our results. Because of the interchangeability between distance education types, it was difficult to compare this study with other research. However, we tried to consider the standard dimensions of this education in our study. In our research environment, online learning has been used.