Abstract
Background:
In medical education, determining the strengths, weaknesses, desirability, and success of training courses from the perspective of its stakeholders is of particular importance because it can be the basis for subsequent decisions.Objectives:
This cross-sectional research investigated the virtual summer semester from the perspectives of its many stakeholders.Methods:
We conducted a multi-methods cross-sectional study. First, remote qualitative interviews were conducted with teachers and heads of departments to explain the features of the virtual summer semester course from their perspective. Twelve teachers and heads of departments participated in qualitative interviews using the intensity sampling method. A matrix was created in Microsoft Word according to the qualitative interview questions (web and telephone), and the answers were categorized based on thematic review. In the second phase, a mini literature review was conducted to develop the desirability and success indicators of online learning courses using questionnaires and checklists. In the third phase, a questionnaire-based survey was conducted among students (82 people). In the last phase, a checklist-based survey was performed among various stakeholders (100 people). Quantitative and qualitative data about the virtual summer semester course have been collected from 194 individuals.Results:
Based on the thematic review of remote qualitative interviews and created matrix, strengths, weaknesses, and suggestions for improving this course were categorized. Based on descriptive statistics, the mean and SD of the age of teachers and heads of departments participating in the interview was (52.5 ± 8.68), and the mean and SD of the age of students participating in the questionnaire-based survey was (22.10 ± 2.78). In addition, the mean and SD of the age of students, teachers, heads of departments, and course executive team participating in the checklist-based survey were respectively (22.5 ± 3.5), (49.1 ± 7.5), (54.0 ± 2.8) and (34.3 ± 4.9). The findings of the questionnaire-based survey revealed that about 91% of the students have a positive attitude towards the course. In this regard, the average score of the students on all items about the desirability of the course was higher than the hypothetical average (P = 0.048). In addition, the quantitative findings indicated that all the stakeholders had evaluated the course as successful in terms of the organizational capacity of Smart University of Medical Sciences, pedagogy and educational effectiveness, and sufficient and capable human resources. In this regard, the average score of the course success factors differs from stakeholder to stakeholder for the organizational capacity (P = 0.004) and for the two components of pedagogy and educational effectiveness (P = 0.035) and sufficient and competent human resources (P = 0.043).Conclusions:
According to the results and the conditions of the COVID-19 pandemic and its effect on medical education, these short-term and intensive structured virtual courses can be expanded and developed into regular semesters. The course's sustainability can be maintained by continual improvement.Keywords
Medical Education Online Course Summer Semester Distance Education Course Evaluation Multi-methods
1. Background
With the advent of information and communication technology (ICT), fundamental changes have occurred in higher education's processes, methods, and products. Many universities have adopted e-learning to deliver their courses and are attempting to increase the efficacy of emerging technologies in educational activities (1). In the fourth and fifth generations of distance education, e-learning has enabled lifelong learning in any subject, for anybody, at any time, and in any location. There is a growing interest in using virtual and online learning in medical education, beginning with the undergraduate years and extending through residency, fellowship, and continuing medical education (2). In the meantime, with the rise of the phenomena and crisis of COVID-19, an extraordinary and revolutionary change occurred in remote learning, resulting in a paradigm shift toward online learning as an alternative educational approach as the only feasible solution (3, 4).
Studies show that COVID-19 did not only lead to a paradigm shift towards e-learning, but this pandemic has brought good opportunities and achievements for medical education, such as improvements in attitudes to e-learning and adaptation, preventing students' separation from the educational environments, self-learning control, documentation, and monitoring training, increasing perceived usefulness (5).
In response to this global emergency, universities, medical schools, and other higher education institutions have been forced to close university campuses and switch to online learning to avoid the suspension of education. If they have a virtual education system, they should work to grow and enhance it (6).
Studies show that students prefer virtual education formats because of their flexibility and because they facilitate learning by providing opportunities to reflect on course material (7). In addition, studies indicate that e-learning, by increasing students' engagement (8) and self-directed learning opportunities (9), may enhance their learning.
In this regard, e-learning approaches in universities and medical institutions have spread in various ways, with the virtual summer semester course in Iran being one of its manifestations. Virtual summer courses offer opportunities to learn or continue education for students who cannot participate in face-to-face summer courses, or in general it is not possible to design and implement face-to-face summer semester courses as a result of crises such as COVID-19 (10).
Despite the fact that teachers and administrators frequently have a negative attitude toward summer courses and believe that these courses are less effective than regular semesters (11-13), research indicates that online summer semester courses are an effective strategy for creating educational equity and participation among underprivileged students, maintaining student graduation rates, and decreasing dropout rates (14).
In addition, Stephenson-Hunter et al. (15) demonstrated in a mixed-method evaluative study that the participation of undergraduate students in a summer virtual mentorship course during the COVID-19 pandemic significantly strengthened non-cognitive skills, particularly self-efficacy.
In general, the virtual summer semester course was designed and implemented as a large-scale national project for all medical science universities across Iran. While this course has certain advantages like other online courses, its success rate, usefulness, strengths, and weaknesses are particularly important from the point of view of its stakeholders, and the results of this research can be the basis for policy-making and decision-making regarding the implementation of its next courses. Therefore, in this research, the national virtual summer semester course was investigated using a multimethod approach and from the perspective of its numerous stakeholders.
2. Objectives
What are the strengths and weaknesses of the virtual summer semester from the point of view of teachers and heads of departments?
How can the virtual summer semester be improved?
To what extent has the virtual summer semester been desirable and appropriate from the perspective of students?
To what extent has the virtual summer semester been successful from the point of view of various stakeholders?
3. Methods
3.1. Setting and Course Description
Undergraduate medical education in Iran has two pre-clinical courses (including two stages of basic science and semiology or physiopathology) and a clinical course (including two phases of clinical clerkship and internship). The virtual summer semester course in the academic year of 2022 was designed and implemented for 125 lessons by 18 departments at Smart University of Medical Sciences for students of medical sciences universities across Iran.
3.2. Multiple Methods and Data Sources (Triangulation)
While mixed methods research refers to combining at least one quantitative and one qualitative method, the multi-methods research approach is not limited solely (and entirely) to combining quantitative and qualitative methods. Multi-methods research integrates several quantitative methods, several qualitative methods, or a combination of both methods (16) to collect richer data. Teddlie and Tashakkori (17) define the multiple-method approach as a sort of design in which more than one method or more than one worldview is utilized to collect data. Considering that the virtual summer semester course includes several stakeholders (such as students, teachers, heads of departments, managers, and an executive staff) with conflicting interests, the use of several methods and several data sources, in addition to helping to eliminate blind spots in the collected data also balance the interests of different stakeholders (16). Hence, this study employed a combination of approaches (including remote qualitative interviews, a mini literature review, a questionnaire-based survey, and a checklist-based survey) to collect.
3.2.1. Remote Qualitative Interviews (Web-based and Telephone)
To investigate of the virtual summer semester course and suggestions for its improvement from the point of view of teachers and heads of departments, remote qualitative interviews (web-based and telephone) were used. In this phase, the open-ended questions were uploaded to the Porsline, and after its final editing, its link was shared with the teachers and chairpersons of departments via email. In addition, a number of teachers and heads of departments were interviewed by phone. Twelve teachers and heads of departments participated in remote qualitative interviews using the intensity sampling method. The second qualified researcher conducted all interviews. In the telephone interviews, an initial question was asked to establish rapport with the participants, followed by four key questions (Appendix 1 in the Supplementary File). The average telephone interview time for each participant was 25 minutes. A matrix was created in Microsoft Word according to the qualitative interview questions (web and telephone), and the answers were categorized based on thematic review. Strategies such as prolonged engagement with data (three of the authors were directly engaged in the design and implementation of the course), member checking, coding, and categorization based on the matrix, devoting adequate time to data collection and analysis, audit trail, quality assessment of data by two medical education and e-learning specialists, were used for data trustworthiness
3.2.2. Mini Literature Review
In the second phase, a mini-review was conducted to review the literature related to evaluating online and e-learning courses. Persian and English articles from the end of 2019 to the end of 2022 were searched in Google Scholar and PubMed databases. The keywords “Desirability,” “Success,” “Attitude to e-learning”, “e-Learning”, “Online learning”, “Online course”, “Virtual course”, “Distance education”, “Summer course” and terms related to each of these keywords were searched. This phase aimed to develop the desirability and success indicators of online learning courses using questionnaires and checklists.
3.2.3. Survey 1 (Questionnaire)
Based on the second phase, a questionnaire with 8 questions about demographic characteristics and 35 Likert scale questions (very low/1 to very high/5) was designed. Face and content validity was used to determine the questionnaire's validity. Face validity was qualitatively conducted in a 6-member panel consisting of faculty members of Smart University of Medical Sciences with specialties (medical education, e-learning, educational technology, and instructional design). This way, the level of difficulty, the degree of disproportion, the ambiguity of the questions, or the presence of insufficiency in the meanings of the words were checked, and their opinions were applied in the form of minor changes in the questionnaire. The content validity was evaluated quantitatively based on the opinion of experts and by calculating two indices, CVR and CVI. To check these two indicators, the questionnaire was sent to 10 experts (medical education, e-learning, educational technology, and instructional design). The CVR results for 32 questions of this questionnaire (out of 35 questions of the initial questionnaire) were equal to or greater than Lawshe's table number (0.62), and as a result of this process, 3 questions were removed. Therefore, the final questionnaire was compiled with 32 questions (Appendix 2 in the Supplementary File). The results of CVI indicated that all questions except questions 5, 9, and 16 had a score higher than 0.79, and these three questions were revised. Finally, Cronbach's alpha index was used to estimate the internal consistency of this questionnaire, and its value was calculated and confirmed as 0.84 for the total instrument. This questionnaire was designed with the aim of investigating students' attitudes towards the desirability of the course. The desirability of questionnaire items was determined by summing the products of each option's frequency score divided by the total number of responders. For each question, a score between 1 and 2.33 was deemed unfavorable, a score between 2.34 and 3.66 was deemed relatively favorable, and a score between 3.67 and 5 was deemed favorable.
3.2.4. Survey 2 (Checklist)
Based on the second stage, a checklist was designed, including 6 demographic questions and 90 questions with three options (relative success, complete success, and unrelated). Face and content validity were employed to determine its validity. Qualitative face validity was also performed for this checklist, and the necessary changes were applied based on the opinions of the experts present in the panel. Content validity was evaluated quantitatively, based on the opinion of experts (10 people), and by calculating the CVR and CVI indexes for this checklist as well. The CVR results for 82 questions of this checklist (out of 90 questions of the initial checklist) were equal to or greater than Lawshe's table number (0.62), and as a result of this process, 8 questions were removed. Therefore, the final checklist was developed with 82 questions (Appendix 3 in the Supplementary File). The results of CVI indicated that all questions had a score higher than 0.79. Finally, Cronbach's alpha index was used to estimate the internal consistency of this checklist, and its value was calculated and confirmed as 0.82 for the total instrument. The purpose of this checklist was to evaluate the degree of success of the course from the perspectives of various stakeholders (students, teachers, heads of departments, managers, and executive staff of the course).
3.3. Participants, Samples, and Sample Size
In order to select the sample and participants, three types of sampling methods were used.
Table 1 shows the participants/samples and the sampling methods.
Participants, Samples, and Their Selection Methods
| Participants/Samples | Sampling Method | ||
|---|---|---|---|
| Intensity Sampling | Cluster Sampling | Relative Stratified Sampling | |
| Remote qualitative interviews (teachers and heads of departments) | √ | ||
| Survey 1 (students) | √ | ||
| Survey 2 (students, teachers, heads of departments, managers, and executive staff of the course) | √ | ||
In order to collect qualitative data from teachers and heads of departments, purposeful sampling and the method with intensity sampling were used. In fact, in this method, people who showed the phenomenon to a large extent were selected, and their participation rate was at the highest level because of the information they had. In the questionnaire-based survey phase, a multi-stage cluster sampling method was used to select students. In other words, the total population of students participating in the course was considered in terms of departments (clusters), and groups with the largest population were selected; then, among the students who had the most logins in the LMS system in these departments, the study sample was selected. Finally, in the phase of a checklist-based survey, in order to select the sample from among different stakeholders (such as students, teachers, heads of Departments, managers, and executive staff of the course), a relatively stratified sampling method was used. In other words, the sample was selected according to the size of the population of these groups.
Table 2 shows the number of participants and samples separately for each group.
Number of Participants and Samples
| Descriptive Statistics and Participants | No. | The Sum of Each Group (%) |
|---|---|---|
| Teachers and heads of departments in web-based qualitative interview | 7 | 12 (6.2) |
| Teachers and heads of departments in telephone qualitative interview | 5 | |
| Survey 1: | ||
| Students | 82 | 82 (42.3) |
| Survey 2 | ||
| Students | 74 | 100 (51.5) |
| Teachers | 16 | |
| Heads of Departments | 2 | |
| Managers and executive staff of the course | 8 | |
| Total | 194 (100.0) |
As shown in Table 2, data was collected quantitatively and qualitatively from a total of 194 individuals. Sampling in the qualitative phase continued until the theoretical saturation of the data. In the quantitative phase (surveys 1 and 2), in order to estimate the sample size, the study of Motte-Signoret et al. (18) was used as the basis of the work. According to the study by Motte-Signoret et al. (18), only 41% of students and 30% of teachers believed that the e-learning course was successful and that they received or provided virtual training with the level and quality as expected. Furthermore, only one-third of students and teachers have indicated that online education will continue after the end of the COVID-19 crisis. In general, based on this study, the P ratio (0.2) was calculated. By placing this ratio in the sample size formula for prevalence and cross-sectional studies (19, 20), the sample size of 246 people was estimated (d = 0.05, z = 1.96, P = 0.2, q = 0.8).
In total, 82 students participated in (survey 1), 74 students in (survey 2), 18 teachers and heads of departments in (survey 2), and 8 members of the executive staff in (survey 2). Therefore, the response rate was about 74%.
3.4. Data Analysis
The data of this study were analyzed in two quantitative and qualitative levels. The thematic review method was used to analyze the qualitative data. Quantitative data analysis was also performed at two descriptive and inferential levels. To use parametric tests (one-sample T and ANOVA), the assumption of normality of the data and homogeneity of variances of the variables were checked and confirmed.
4. Results
Through multi-methods, study (remote qualitative interview, mini literature review, questionnaire-based survey, and checklist-based survey) and data source (students, teachers, heads of departments, managers, and executive staff of the course), the attitudes and viewpoints of different stakeholders about the virtual summer semester were presented in the form of quantitative and qualitative results.
4.1. Descriptive of Participants/Sample
Table 3 presents the demographic characteristics of the study participants separately for an interview, questionnaire-based survey, and checklist. As can be seen based on the collected data, two of the web-based interview participants did not declare demographic characteristics.
Description of Research Participants
| Variables | No. (%) or Mean ± SD |
|---|---|
| Description of Interview Participants | |
| Gender | |
| Male | 7 (70.0) |
| Female | 3 (30.0) |
| Age | 52.5 ± 8.68 |
| 38 - 43 | 2 (20.0) |
| 44 - 49 | 1 (10.0) |
| 50 - 55 | 3 (30.0) |
| 56 - 61 | 2 (20.0) |
| 62 - 67 | 2 (20.0) |
| Academic rank | |
| Assistant professor | 1 (10.0) |
| Associate professor | 3 (30.0) |
| Professor | 6 (60.0) |
| Total | 10 (100.0) |
| Description of Questionnaire-based Survey Participants | |
| Gender | |
| Male | 44 (53.7) |
| Female | 38 (46.3) |
| Age | 22.10 ± 2.78 |
| 19 - 22 | 57 (69.5) |
| 23 - 26 | 19 (23.2) |
| 27 - 30 | 4 (4.9) |
| 31 - 34 | 1 (1.2) |
| 35 - 38 | 1 (1.2) |
| Distribution by field of study | |
| Medicine | 50 (61.0) |
| Dental | 17 (20.0) |
| Pharmacology | 8 (10.0) |
| Laboratory sciences | 3 (4.0) |
| Midwifery | 1 (1.0) |
| Nursing | 3 (4.0) |
| Distribution by native | |
| Iranian | 80 (97.6) |
| Non-Iranian (foreign) | 2 (2.4) |
| Total | 82 (100.0) |
| Description of Checklist-based Survey Participants and Gender | |
| Students | |
| Male | 35 (47.3) |
| Female | 39 (52.7) |
| Teachers | |
| Male | 7 (43.8) |
| Female | 9 (56.2) |
| Heads of departments | |
| Male | 0 (0.0) |
| Female | 2 (100.0) |
| Course executive team | |
| Male | 1 (12.5) |
| Female | 7 (87.5) |
| Age | |
| Students | 22.5 ± 3.5 |
| 19 - 26 | 64 (86.5) |
| 27 - 34 | 9 (12.2) |
| 35 - 42 | 1 (1.4) |
| Total | 74 (100.0) |
| Teachers | 49.1 ± 7.5 |
| 35 - 42 | 4 (25.0) |
| 43 - 50 | 4 (25.0) |
| 51 - 58 | 5 (31.2) |
| 59 - 66 | 2 (12.5) |
| Total | 15 (93.7) |
| Missing | 1 (6.3) |
| Heads of departments | 54.0 ± 2.8 |
| 51 - 58 | 2 (100.0) |
| Course executive team | 34.3 ± 4.9 |
| 19 - 26 | 1 (12.5) |
| 27 - 34 | 2 (25.0) |
| 35 - 42 | 5 (62.5) |
| Total | 8 (100.0 |
| Academic rank of teachers and heads of departments | |
| Instructor | 1 (5.5) |
| Assistant professor | 6 (33.3) |
| Associate professor | 8 (44.5) |
| Professor | 3 (16.7) |
4.2. Qualitative Findings
Using web and telephone interviews, the virtual summer semester course was investigated from the perspective of teachers and heads of departments. Based on the thematic review of remote qualitative interviews and the created matrix, strengths, weaknesses, and suggestions for improving this course were categorized (see Table 4).
Data Analysis of Remote Qualitative Interviews
| Main Categories | Codes Derived from Web-based and Telephone Interviews |
|---|---|
| Strengths of the virtual summer semester course | - Underlie educational equity |
| - Compensating students' academic backwardness | |
| - The necessity of such courses due to epidemics such as COVID-19 | |
| - Students' access to standard educational contents | |
| - Removing geographic distance | |
| - Developing of self-directed learners | |
| - Diversity in offering courses and paying attention to students' needs | |
| - Using experienced and competent teachers in the course | |
| - Reducing educational costs | |
| - Applying diverse teaching methods that suit the characteristics of the course and students | |
| Weaknesses of the virtual summer semester course | - Poor teacher-student interaction |
| - Lack of necessary infrastructure to hold classes online (teaching-learning synchronous) | |
| - Lack of sufficient facilities and equipment to deliver practical and skill classes | |
| - Weakness in student assessment (assessment needs a better mechanism) | |
| Suggestions for improving the virtual summer semester course | - developing intrinsic and extrinsic motivation in teachers to more engage in the course |
| - Timely notification for planning in department(s) | |
| - Providing the necessary infrastructure to hold one-third of the classes online (teaching-learning synchronous) | |
| - Student support system (educational/pedagogical and technical) | |
| - Revision of contents in each course | |
| - Modifying the student assessment (assessment should not be just tests and exams) |
Based on the data analysis of remote qualitative interviews, the responses of teachers and department heads were presented in three categories: the course's strengths, its weaknesses, and improvement ideas.
4.3. Quantitative Findings
Based on the data collected from the questionnaire-based survey, students' attitudes toward the course were placed on a three-spectrum continuum (unfavorable, relatively favorable, and favorable). Table 5 shows an example of questions with descriptive statistics.
An Example of the Percentage of Students' Responses and the Favorability Level of the Course from Their Point of View
| Questions Related to the Desirability of the Course from the Perspective of Students | Answers to the Questions (%) | Mean | Favorability Level | ||||
|---|---|---|---|---|---|---|---|
| Very Low | Low | Moderate | High | Very High | |||
| 1. To what extent were your needs addressed in the design and implementation of the summer semester course? | 12.2 | 11.0 | 34.1 | 30.5 | 12.2 | 3.20 | Relatively favorable |
| 2. To what extent were the students' facilities considered in developing the goals of the virtual summer semester course? | 17.1 | 6.1 | 32.9 | 30.5 | 13.4 | 3.17 | Relatively favorable |
| 3. To what extent were the objectives of the virtual summer semester course clear? | 12.2 | 8.5 | 29.3 | 35.4 | 14.6 | 3.32 | Relatively favorable |
| 4. To what extent was the content of the courses presented in the virtual summer semester course complete and comprehensive? | 7.3 | 3.7 | 24.4 | 42.7 | 22.0 | 3.68 | Favorable |
| 5. To what extent was the virtual summer semester course effective in your growth and learning? | 9.8 | 11.0 | 23.2 | 34.1 | 22.0 | 3.48 | Relatively favorable |
In general, based on the results of the questionnaire-based survey, approximately 91% of the students have a positive attitude towards this course. The students evaluated the virtual summer semester course as favorable in terms of the comprehensiveness of the course content. In addition, most students agreed with the importance of continuing the course and occasionally changing the schedule for subsequent courses. In order to study the statistical significance level of the course's favorability from the student's point of view, the one-sample T-test was used. The results of the one-sample T-test indicate that the sample mean is greater than the population mean (P = 0.048).
Another objective of this research was to investigate the course's success based on the checklist tool from the perspective of different stakeholders. In this regard, the descriptive findings showed that all stakeholders had evaluated the course as successful in terms of the organizational capacity of Smart University of Medical Sciences, pedagogy and educational effectiveness, and human resources engaged in the course. Thus, the ANOVA test was used to compare the averages of these three components.
The statistics presented in Table 6 show that the average score of organizational capacity of Smart University of Medical Sciences (P = 0.004), pedagogy and educational effectiveness (P = 0.035), and human resources engaged in the course (P = 0.043) is different based on multiple stakeholders.
ANOVA Test Regarding Course Success
| Variables | No. | Mean | P-Value |
|---|---|---|---|
| Organizational capacity of Smart University of Medical Sciences | 0.004 a | ||
| Students | 74 | 23.77 | |
| Teachers | 16 | 18.56 | |
| Heads of departments | 2 | 17.50 | |
| Couse executive staffs | 8 | 29.25 | |
| Pedagogy and educational effectiveness | 0.035 b | ||
| Students | 74 | 50.67 | |
| Teachers | 16 | 38.12 | |
| Heads of departments | 2 | 30.0 | |
| Couse executive staffs | 8 | 60.37 | |
| Human resources engaged in the course | 0.043 b | ||
| Students | 74 | 21.31 | |
| Teachers | 16 | 18.12 | |
| Heads of departments | 2 | 18.50 | |
| Course executive staffs | 8 | 33.25 |
5. Discussion
The virtual summer semester course for the academic year of 2022 was designed and implemented for 125 lessons by 18 departments at Smart University of Medical Sciences for students of medical sciences universities across Iran. This evaluative study evaluated this course with a multimethod approach and from the perspective of its various stakeholders. In order to evaluate this course from the point of view of teachers, heads of departments, students, and executive staff, three tools, including remote qualitative interviews and a Likert scale questionnaire and checklist, were developed.
Based on the thematic analysis of the responses of the teachers and heads of departments to the remote qualitative interviews (web-based and telephone), the features of the virtual summer semester were categorized into three themes: strengths, weaknesses, and suggestions for improvement.
In addition, based on the data collected from the questionnaire-based survey, the status of the virtual summer semester from the student's point of view was scored on a continuum (unfavorable, relatively favorable, and favorable). Based on the analysis of the collected data, approximately 91% of the students participating in the research have a positive attitude toward this course. In this regard, the average score of the students in all the questions related to the favorability of the course was higher than the hypothetical average (P = 0.048).
Finally, based on the data collected from the checklist tool, all the stakeholders evaluated the virtual summer semester course favorably in terms of the organizational capacity of the Smart Medical Sciences University (allocation of sufficient materials and resources, commitment and support to provide a quality course, etc.), pedagogy and educational effectiveness (clear design of the course, creating diverse learning opportunities, constructive alignment between goals, teaching-learning methods, assessment, etc.) and sufficient and capable human resources (teachers, managers and executive staff). In this regard, the average score of the course's success components differed according to the perspective of different stakeholders for the organizational capacity (P = 0.004), for the pedagogy and educational effectiveness (P = 0.035), and sufficient and capable human resources components (P = 0.043).
In general, the COVID-19 pandemic has accelerated the implementation of e-learning, and e-learning in education and higher education, including universities and schools of medical sciences, has become a double necessity in this era. These trainings, which are typically delivered in a blended format, can be utilized by students in clinical, basic sciences, and post-graduate courses and provide both theoretical and clinical training. They offer several benefits, including the ability for students to access the training at the desired time, lower costs, up-to-date educational materials, flexible learning, more freedom for students, and the ability to keep their jobs and earn money (21).
Although the virtual summer semester was introduced as one kind of virtual education before the coronavirus pandemic, it was created and implemented with increased emphasis and significance in this era because of the global paradigm shift toward virtual education and e-learning. Yet, the relevance of evaluating this course is no less than its design and implementation. Hence, the focus of this study was on evaluating this course. Evaluating various training courses is a prerequisite for improving and enhancing their quality and should be considered using scientific and systematic methods (22).
Concerning the qualitative findings of this study, some of the emergent results support the findings in the literature. For example, participants in this study cited flexibility as one of the course's strengths, which is consistent with the findings of Zheng et al. (23), Amir et al. (24), and Dost et al. (25). The participants also mentioned educational equity and compensating students' academic backwardness as one of the course's strengths, which aligns with the findings of Bentley's study (14). Similarly, in our study, participants identified the lack of teacher-student interaction as one of the virtual summer semester course's disadvantages, which is consistent with the findings of studies by Wang et al. (26), Zheng et al. (23), Abbasi et al. (27) and Amir et al. (24).
In summary, even though the qualitative findings indicated shortcomings in the virtual summer semester course, the results of the questionnaire-based survey among various stakeholders reflect the course's success. Meanwhile, the findings from the checklist-based survey among students (as the main stakeholders) show that a very high percentage of them expressed satisfaction with this course and evaluated it from relatively favorable to favorable, which is consistent with some studies in the literature. For example, Zheng et al. (23) investigated students' perceived effectiveness of online courses during the summer quarter of 2020. Also, in this study, the scores of students exposed to the online course during the summer quarter of 2020 were compared with the control group who had taken the same course face-to-face during the summer quarter of 2019 before the COVID-19 pandemic. Their study showed that students generally had a positive attitude about online learning during the pandemic, and most wanted to continue learning online after the pandemic. In addition, according to the findings of this study, the online group received as many or more a grades during the summer quarter of 2020 compared to the identical face-to-face group during the summer quarter of 2019. Meanwhile, Fischer et al. (10) concluded that students' performance in online courses was slightly lower compared to face-to-face courses, which is contrary to the results of our research.
Furthermore, similar to our findings, Goppert & Pfost's (28) research revealed that students experienced less worry and more enjoyment during online summer semester courses. Much research has been conducted in the Corona era regarding the satisfaction, attitude, and effectiveness of online courses (excluding summer semester virtual courses). The results of our research are consistent with those of AlQhtani et al. (29), Sadid-Zadeh et al. (30), Rad et al. (31), Agarwal & Kaushik (32), Khalil et al. (33), and partially aligns with the study of Hameed et al. (34).
This study has implications for practice and policy based on findings such as a positive attitude toward the course, satisfaction with the course, and the success and effectiveness of the course obtained from the qualitative interview, questionnaire, and checklist. Among them, policymakers and planners can expand virtual education and e-learning to long-term courses and develop regular and annual term courses in blended learning. Secondly, based on the implementation of this course in a wide geographic distribution (throughout the country) and its effectiveness, it can be stated that these short-term and intensive e-learning courses can accommodate large groups of students with diverse individual and cultural backgrounds. Hence, policymakers can be somewhat relieved in this regard. Based on this experience and the lessons gained, they can extend development-focused short-term e-learning courses at the level of continuous education. Finally, according to the results of this study, to sustainably improve the virtual summer semester course, policymakers can integrate it with a blended learning approach into the undergraduate curriculum.
There were some limitations in our research. First, because the geographical distribution of the participants and the study sample was so extensive and spanned the entire country, we did not have direct access to them; thus, the response rate in some universities was slow and poor. Second, our study's sample and participants came from different groups of stakeholders, making data collecting challenging and time-consuming. The other limitation is that two data components were collected via a survey (questionnaire and checklist) and self-report. Hence, it is likely that participants selected the answer that was most advantageous to them, compromising the data's reliability. In conclusion, we recommend that future studies examine students' performance, academic success, and grade comparisons in virtual summer semester courses to assess their usefulness. In addition, longitudinal studies with a concentration on one area of the educational course, such as the quality of instruction, assessment, and the learning of students etc., can provide the basis for future research.
5.1. Conclusions
In conclusion, although the results of this study revealed weaknesses associated with the virtual summer semester course, the findings indicated that students have a positive attitude toward this course, and various stakeholders have deemed the implementation of this course a success. Thus, based on the results and the conditions of the COVID-19 pandemic and its impact on medical education, such short-term and intensive, structured e-learning and virtual courses can be expanded and developed into regular semesters, and the course's sustainability can be ensured through continuous improvement.
Acknowledgements
References
-
1.
Andone L, Sireteanu NA. Strategies for technology-based learning in higher education. FedUni J Higher Educ. 2009;4(1):31-42.
-
2.
Lewis KO, Cidon MJ, Seto TL, Chen H, Mahan JD. Leveraging e-learning in medical education. Curr Probl Pediatr Adolesc Health Care. 2014;44(6):150-63. [PubMed ID: 24981664]. https://doi.org/10.1016/j.cppeds.2014.01.004.
-
3.
Allen IE, Seaman J. Grade Level: Tracking Online Education in the United States. ERIC; 2015.
-
4.
Ahmed H, Allaf M, Elghazaly H. COVID-19 and medical education. Lancet Infect Dis. 2020;20(7):777-8. [PubMed ID: 32213335]. [PubMed Central ID: PMC7270510]. https://doi.org/10.1016/S1473-3099(20)30226-7.
-
5.
Hayat AA, Keshavarzi MH, Zare S, Bazrafcan L, Rezaee R, Faghihi SA, et al. Challenges and opportunities from the COVID-19 pandemic in medical education: a qualitative study. BMC Med Educ. 2021;21(1):247. [PubMed ID: 33926439]. [PubMed Central ID: PMC8082480]. https://doi.org/10.1186/s12909-021-02682-z.
-
6.
Farokhi M. A Survey of Tehran Refah College Students' Evaluation of Virtual Education in the Corona Period. Relig Commun. 2021;28(59):274-47.
-
7.
Dyrbye L, Cumyn A, Day H, Heflin M. A qualitative study of physicians' experiences with online learning in a masters degree program: benefits, challenges, and proposed solutions. Med Teach. 2009;31(2):e40-6. [PubMed ID: 19330663]. https://doi.org/10.1080/01421590802366129.
-
8.
Liaw S, Huang H, Chen G. An activity-theoretical approach to investigate learners’ factors toward e-learning systems. Comput Hum Behav. 2007;23(4):1906-20. https://doi.org/10.1016/j.chb.2006.02.002.
-
9.
Leidner DE, Jarvenpaa SL. The Use of Information Technology to Enhance Management School Education: A Theoretical View. MIS Quarterly. 1995;19(3):265. https://doi.org/10.2307/249596.
-
10.
Fischer C, Xu D, Rodriguez F, Denaro K, Warschauer M. Data on online and face-to-face course enrollments in a public research university during summer terms. Data Brief. 2020;29:105320. [PubMed ID: 32181291]. [PubMed Central ID: PMC7062920]. https://doi.org/10.1016/j.dib.2020.105320.
-
11.
Scott PA. Attributes of high‐quality intensive courses. New Dir Adult Contin Educ. 2003;2003(97):29-38. https://doi.org/10.1002/ace.86.
-
12.
Kretovics MA, Crowe AR, Hyun E. A Study of Faculty Perceptions of Summer Compressed Course Teaching. Innov High Educ. 2005;30(1):37-51. https://doi.org/10.1007/s10755-005-3295-1.
-
13.
Daniel EL. A review of time-shortened courses across disciplines. Coll Stud J. 2000;34(2).
-
14.
Bentley ML. An evaluation of an online high school summer credit recovery program to maintain Virginia on-time graduation. The College of William and Mary; 2019.
-
15.
Stephenson-Hunter C, Franco S, Martinez A, Strelnick AH. Virtual Summer Undergraduate Mentorship Program for Students Underrepresented in Medicine Yields Significant Increases in Self-Efficacy Measurements During COVID-19 Pandemic: A Mixed Methods Evaluation. Health Equity. 2021;5(1):697-706. [PubMed ID: 34909539]. [PubMed Central ID: PMC8665814]. https://doi.org/10.1089/heq.2021.0060.
-
16.
Bauer C. Multi-Method Evaluation: Leveraging Multiple Methods to Answer What You Were Looking For. Proceedings of the 2020 Conference on Human Information Interaction and Retrieval. 2020. p. 472-4.
-
17.
Teddlie C, Tashakkori A. Major issues and controversies in the use of mixed methods in the social and behavioural sciences. Handbook of mixed methods in social and behavioural research. London, UK: Sage; 2003.
-
18.
Motte-Signoret E, Labbe A, Benoist G, Linglart A, Gajdos V, Lapillonne A. Perception of medical education by learners and teachers during the COVID-19 pandemic: a cross-sectional survey of online teaching. Med Educ Online. 2021;26(1):1919042. [PubMed ID: 33871308]. [PubMed Central ID: PMC8079026]. https://doi.org/10.1080/10872981.2021.1919042.
-
19.
Daniel WW. Biostatistics: a foundation for analysis in the health sciences. Wiley; 1978.
-
20.
Pourhoseingholi MA, Vahedi M, Rahimzadeh M. Sample size calculation in medical studies. Gastroenterol Hepatol Bed Bench. 2013;6(1):14-7. [PubMed ID: 24834239]. [PubMed Central ID: PMC4017493].
-
21.
Simpson O. Supporting students for success in online and distance education. New York, USA: Routledge; 2013.
-
22.
Campbell SM, Braspenning J, Hutchinson A, Marshall MN. Research methods used in developing and applying quality indicators in primary care. BMJ. 2003;326(7393):816-9. [PubMed ID: 12689983]. [PubMed Central ID: PMC1125721]. https://doi.org/10.1136/bmj.326.7393.816.
-
23.
Zheng M, Bender D, Lyon C. Online learning during COVID-19 produced equivalent or better student course performance as compared with pre-pandemic: empirical evidence from a school-wide comparative study. BMC Med Educ. 2021;21(1):495. [PubMed ID: 34530828]. [PubMed Central ID: PMC8443899]. https://doi.org/10.1186/s12909-021-02909-z.
-
24.
Amir LR, Tanti I, Maharani DA, Wimardhani YS, Julia V, Sulijaya B, et al. Student perspective of classroom and distance learning during COVID-19 pandemic in the undergraduate dental study program Universitas Indonesia. BMC Med Educ. 2020;20(1):392. [PubMed ID: 33121488]. [PubMed Central ID: PMC7594975]. https://doi.org/10.1186/s12909-020-02312-0.
-
25.
Dost S, Hossain A, Shehab M, Abdelwahed A, Al-Nusair L. Perceptions of medical students towards online teaching during the COVID-19 pandemic: a national cross-sectional survey of 2721 UK medical students. BMJ Open. 2020;10(11). e042378. [PubMed ID: 33154063]. [PubMed Central ID: PMC7646323]. https://doi.org/10.1136/bmjopen-2020-042378.
-
26.
Wang K, Zhang L, Ye L. A nationwide survey of online teaching strategies in dental education in China. J Dent Educ. 2021;85(2):128-34. [PubMed ID: 32954532]. [PubMed Central ID: PMC7537095]. https://doi.org/10.1002/jdd.12413.
-
27.
Abbasi S, Ayoob T, Malik A, Memon SI. Perceptions of students regarding E-learning during COVID-19 at a private medical college. Pak J Med Sci. 2020;36(COVID19-S4):S57-61. [PubMed ID: 32582315]. [PubMed Central ID: PMC7306963]. https://doi.org/10.12669/pjms.36.COVID19-S4.2766.
-
28.
Goppert SA, Pfost M. Undergraduate Students' Perceived Stress Levels in Summer Term 2020 - A Comparison to Preceding Academic Terms. Front Psychol. 2021;12:672783. [PubMed ID: 34149568]. [PubMed Central ID: PMC8206474]. https://doi.org/10.3389/fpsyg.2021.672783.
-
29.
AlQhtani A, AlSwedan N, Almulhim A, Aladwan R, Alessa Y, AlQhtani K, et al. Online versus classroom teaching for medical students during COVID-19: measuring effectiveness and satisfaction. BMC Med Educ. 2021;21(1):452. [PubMed ID: 34454493]. [PubMed Central ID: PMC8397601]. https://doi.org/10.1186/s12909-021-02888-1.
-
30.
Sadid-Zadeh R, Wee A, Li R, Somogyi-Ganss E. Audience and Presenter Comparison of Live Web-Based Lectures and Traditional Classroom Lectures During the COVID-19 Pandemic. J Prosthodont. 2021;30(5):412-9. [PubMed ID: 33283911]. https://doi.org/10.1111/jopr.13301.
-
31.
Rad FA, Otaki F, Baqain Z, Zary N, Al-Halabi M. Rapid transition to distance learning due to COVID-19: Perceptions of postgraduate dental learners and instructors. PLoS One. 2021;16(2). e0246584. [PubMed ID: 33556131]. [PubMed Central ID: PMC7870061]. https://doi.org/10.1371/journal.pone.0246584.
-
32.
Agarwal S, Kaushik JS. Student's Perception of Online Learning during COVID Pandemic. Indian J Pediatr. 2020;87(7):554. [PubMed ID: 32385779]. [PubMed Central ID: PMC7205599]. https://doi.org/10.1007/s12098-020-03327-7.
-
33.
Khalil R, Mansour AE, Fadda WA, Almisnid K, Aldamegh M, Al-Nafeesah A, et al. The sudden transition to synchronized online learning during the COVID-19 pandemic in Saudi Arabia: a qualitative study exploring medical students' perspectives. BMC Med Educ. 2020;20(1):285. [PubMed ID: 32859188]. [PubMed Central ID: PMC7453686]. https://doi.org/10.1186/s12909-020-02208-z.
-
34.
Hameed T, Husain M, Jain SK, Singh CB, Khan S. Online Medical Teaching in COVID-19 Era: Experience and Perception of Undergraduate Students. Maedica (Bucur). 2020;15(4):440-4. [PubMed ID: 33603900]. [PubMed Central ID: PMC7879367]. https://doi.org/10.26574/maedica.2020.15.4.440.