1. Background
Today, the increasing development of science and technology has led to the formation of different organizational structures. In such situations, a successful organization is one that progresses and develops according to new knowledge and advanced technology (1). Manpower is the most important element in this regard, as the efficiency of the organization depends on the proper functioning of these forces. The mere training and implementation of training courses cannot help the organization achieve its goals. Training should be based on scientific principles and methods to meet existing needs. A comprehensive evaluation can show us the effectiveness of the results; evaluation generates feedback that can be used to determine whether the training has been effective in achieving the desired goals (2). What is extremely important in the design and implementation of electronic education is to evaluate the effectiveness of this kind of training to be able to design supplementary and desired trainings from the obtained results. Accurate determination of the effects and consequences of a training course on participants and determining how they perform when they return to work in the organization is a complex and difficult process. Sometimes, it is based on the mental judgments of those who put a lot of effort into preparing and executing an apparently successful training course, but pay less attention to the effects and scientific results of the training course (3).
The extensive use of new technologies and, consequently, the need for more training and higher costs have led organizations to use new training solutions. One of these solutions is e-learning or virtual learning (4). With the beginning of the age of information technology, educational systems were among the first fields to undergo major changes, and the need to revise traditional teaching methods and use new and active learning methods has been felt by policymakers in educational systems (5). The purpose of e-learning is to improve the knowledge and skills of the workforce by using up-to-date and cost-effective programs. There are various and complex topics in the field of medical sciences that need to be properly taught. One of these topics is epilepsy and how to manage it in children. Seizures are an attack phenomenon caused by abnormal, excessive, and simultaneous activity of neurons in the central nervous system (CNS). Depending on the distribution of the discharges, this abnormal CNS activity can manifest itself in a variety of ways, from intense seizure activity to subtle phenomena that are not readily apparent to the observer. Although various factors affect the incidence and prevalence of seizures, about 5 - 10% of the population will have at least one seizure, with the maximum incidence occurring in early childhood and late adulthood (6).
Epilepsy describes a condition in which a person experiences recurrent seizures caused by a chronic underlying process. Epilepsy is more of a clinical phenomenon than a single disease because there are many forms and causes of epilepsy. However, there are various epilepsy syndromes with distinct clinical and pathological features, suggesting specific underlying etiologies. Determining the type of seizure that has occurred is essential to focus on the diagnostic approach to specific etiologies, select appropriate treatment, and obtain potentially vital prognostic information. In 1981, the International League Against Epilepsy (ILAE) released a modified version of the epilepsy classification system, which was reviewed and updated in 2017. Mastery of the correct use of guideline indicators by pediatric neurologists can improve the diagnosis and treatment of patients.
Among the different teaching methods in this research, electronic education was used, and its effectiveness was evaluated based on Kirkpatrick's model to determine its strengths and weaknesses. Donald Kirkpatrick’s model is one of the most well-known models for evaluating and measuring human resource management (7). This model evaluates in-service training at four levels: Reaction, learning, behavior, and outcomes (8). There are three reasons for this evaluation: The importance and existence of the philosophy and mission of the education unit, the decision to continue the training program, and the determination of the effectiveness and improvement of the training program (9, 10), because an instructional program is valuable when it shows documented and reliable evidence of the effects of training on learners' behavior change and performance (11).
Effectiveness is achieved when learners' performance changes in the workplace and their learning is transferred to the real environment (12). Therefore, in this model, effectiveness evaluation is measured during the training period (13). Training alone is not useful unless evaluated (14). Unfortunately, in our country, evaluations of training courses are often done simply at the first or second level of this model, which only reflects success and effectiveness in the initial stages. However, at higher levels, fewer educational effects are observed (15).
2. Objectives
In this regard, the present study was conducted to evaluate the effectiveness of an e-learning course on epilepsy classification guidelines (ILAE 2017) on the performance of pediatric neurologists. Although much research exists on the advantages and disadvantages of e-learning, there is no research on epilepsy guideline education for pediatric specialists.
3. Methods
This is a descriptive comparative study conducted in 2022. The study population comprised pediatric neurologists from universities of medical sciences across the country. Inclusion criteria included participation in the course during the semester, availability, and willingness to participate in the study. Participants entered the study according to their personal preference and provided informed consent, and those who were absent from two or more sessions on the relevant topics were excluded from the study. The study involved participation in the e-learning course on epilepsy classification according to the guidelines (ILAE 2017). The sample size was calculated using Cochran's formula, resulting in 74 participants. Of these, 19 did not meet the inclusion criteria, and 55 physicians were studied.
The steps of the research were as follows: First, the educational multimedia content of the guidelines for classifying epilepsy in the treatment of various seizures in children and adults (ILAE 2017) was prepared by the researcher and approved by pediatric neurology professors. The content was uploaded to the e-learning management system of Tehran University of Medical Sciences at http://tumsnavid.vums.ac.ir. The participants' information was entered into the system, and the condition for accessing the educational content was completing a 15-item, four-choice MCQ test on seizure classification (ILAE 2017). The face and content validity of this test were verified by pediatric neurology professors and medical education specialists, and the reliability coefficient was calculated using Cronbach’s alpha, which was 0.96.
After the educational intervention, the Kirkpatrick model was used to evaluate the e-learning course at three levels: Reaction, learning, and behavior. At the first level (reaction), the standard questionnaire of Zolfaghari et al. (16) was used to assess participant satisfaction, consisting of 40 questions on demographic information and evaluation indicators of the e-learning course. This questionnaire used a 5-point Likert Scale, with a score of one indicating very low and five indicating very high satisfaction. The Cronbach's alpha coefficient of this questionnaire was 0.83. The participants completed this questionnaire at the end of the course. At the second and third levels (learning and behavioral changes), the MCQ test was retaken to determine participants' skill levels in applying the guidelines in diagnosis, treatment, and follow-up. To analyze the data, descriptive statistics (frequency/percentage, mean, standard deviation, and variation range) were used, along with paired t-tests and linear regression (or nonparametric equivalents depending on the data distribution), with a significance level of 0.05. SPSS version 22 statistical software was used for analysis.
3.1. Ethical Considerations
This study was approved by the ethics committee of Tehran University of Medical Sciences under the code of ethics IR.TUMS.CHMC.REC.1398.009. Before the intervention, participants provided written informed consent. At the end of the study, the results were shared with all participants.
4. Results
The study showed that 22 women (40%) and 33 men (60%) participated. The frequency and demographic characteristics are shown in Table 1.
Table 1.Frequency and Demographic Characteristics of the Participants
| Variables | No. (%) |
|---|---|
| Age (y) | |
| 32 - 25 | 2 (3.6) |
| 40 - 33 | 14 (25.5) |
| More than 40 | 39 (70.9) |
| Marital status | |
| Married | 48 (87.3) |
| Single | 7 (12.7) |
| Gender | |
| Male | 33 (60) |
| Female | 22 (40) |
The first research question investigates the participants' reactions to the e-learning course. The results showed that most participants were satisfied with the e-course (Table 2).
Table 2.Evaluation Indicators of the e-learning Course
| Characteristic | Very High (%) | High (%) | Average (%) | Low (%) | Very Low (%) |
|---|---|---|---|---|---|
| Satisfaction with the educational content | |||||
| Fit the content to the needs of the learners | 56 | 40 | 4 | 0.0 | 0.0 |
| Match the content with the objectives of the course | 50 | 47 | 3 | 0.0 | 0.0 |
| Proportion of content quantity with the course unit | 54 | 43 | 3 | 0.0 | 0.0 |
| Content organization | 48 | 51 | 1 | 0.0 | 0.0 |
| Up-to-date course content and quality of course content | 52 | 47 | 1 | 0.0 | 0.0 |
| Proper content arrangement | 52 | 48 | 0 | 0.0 | 0.0 |
| Efficiency of the e-learning infrastructure | |||||
| Ease of system use | 50 | 47 | 3 | 0.0 | 0.0 |
| Speed of access to the system | 48 | 51 | 1 | 0.0 | 0.0 |
| Degree of interactivity of the system | 52 | 47 | 1 | 0.0 | 0.0 |
| Attractiveness and semblance of system pages | 42 | 54 | 3 | 0.0 | 0.0 |
| Effectiveness of the educational content organization | |||||
| Relevance of course resources | 52 | 48 | 0 | 0.0 | 0.0 |
| Access to course materials | 48 | 51 | 1 | 0.0 | 0.0 |
| Possibility to review previous information | 52 | 47 | 1 | 0.0 | 0.0 |
| Effectiveness of the support aspects | |||||
| Availability of a teacher for guidance and advice | 48 | 51 | 1 | 0.0 | 0.0 |
| Existence of face-to-face meetings to solve academic problems | 30 | 32 | 20 | 15 | 3 |
| User technical support | 51 | 43 | 6 | 0.0 | 0.0 |
| Provide instructions for using the system | 48 | 41 | 11 | 0.0 | 0.0 |
| Perform the necessary notifications via email, message, etc. | 40 | 45 | 12 | ||
| Overall satisfaction | |||||
| Overall satisfaction with LMS-based education | 48 | 51 | 1 | ||
| How well are your expectations met during the semester by virtual education? | 46 | 44 | 10 |
To determine the effectiveness of e-learning on children's brain and neurology subspecialties, the correct response rate to each question before and after the course was compared, and the Wilcoxon Signed Ranks Test was used for general analysis. The effect of e-learning on epilepsy classification guidelines according to the ILAE 2017 guidelines was investigated in relation to the performance of neurologists treating children and adults, using the Wilcoxon Signed Ranks Test. The results are shown in Table 3.
Table 3.Comparison of Answers to Questions Before and After the Educational Intervention
a Total correct answers after the intervention < Total correct answers before the intervention.
b Total correct answers after intervention > Total correct answers before intervention.
c Total correct answers after intervention = Total correct answers before intervention.
The significance level between the results before and after the intervention is shown in Table 4. Our findings indicated that the e-learning of epilepsy classification guidelines according to the ILAE 2017 guidelines significantly improved the performance of pediatric neurologists (P-value ˂ 0.001).
5. Discussion
Based on the results at the reaction level, there was a high level of satisfaction in the areas of educational content, educational infrastructure, organizing educational content, and support aspects. In general, it can be said that most participants were completely satisfied with their participation in this course and evaluated it as useful and effective. The goal of educational training is to improve workplace performance; however, limited efforts have been made to confirm how this is done. Burke and Hutchins showed that the role of education in personal and organizational development is less than 50% (17). Training without usability leads to a waste of time and resources in the intended profession. Therefore, it is very important to understand how training can provide the desired performance results. Vaughan and also Garrison suggested that not only does the combination of electronic and face-to-face methods increase learning, but it also enhances interaction and satisfaction in this method (18).
A study conducted in 2020 comparing the effectiveness of electronic and traditional teaching methods for neurology topics among neurology residents revealed that the electronic method resulted in significantly higher learning rates. Additionally, all participants expressed a preference for electronic education over traditional methods (19). A recent study on the continuing education of neurology residents demonstrated that e-learning significantly enhances learning efficiency and reduces costs. Compared to traditional teaching methods, e-learning engages learners more effectively by incorporating videos and images, thereby simplifying the comprehension of intricate neurology topics for young neurologists. Furthermore, in clinical settings, e-learning bridges knowledge and skill gaps through virtual patient interactions and instructional videos on examination techniques and pathological indicators. E-learning is also better equipped to cater to individual learning needs, making it a preferred option for ongoing education among young neurologists (20).
Thiele acknowledges that learners have access to more information through the e-learning approach, take responsibility for their own learning, and can access instructional content whenever they wish, thus facilitating the application of the e-learning method (21). This finding is also consistent with Buckley’s results; he believes that the ease of access to educational content in e-learning methods leads to increased learner satisfaction (22). However, contrary to the results of the present study, in many studies conducted in recent decades, the rate of learning (mean scores) in the "electronic method" is reported to be similar to the "face-to-face method" (23-27). While learners’ learning rates are reported equally in both methods and learners are satisfied with both, a longitudinal study with a control group showed that e-learning is 19% more effective than face-to-face training, which confirms the results of the present study (28).
In 2016, a study by Ghareeb et al. investigated the impact of e-learning on the clinical performance of obstetricians at Illinois General Medical University. The results showed that e-learning can be an effective tool for improving resident clinical performance, which confirms the results of our study (29). Contrary to the findings of the present study, Hugenholtz et al. conducted a study comparing electronic and traditional methods in medical education. Their results showed that both e-learning and lecture-based methods were equally effective in increasing learners’ knowledge, with no statistically significant difference between the two groups (30).
The classification of epilepsy has evolved dramatically since its inception in the 1960s. Frequent revisions indicate progress in recognizing phenotypic patterns and basic mechanisms based on clinical research. These insights, which depend on various aspects of patient care, have led to advances in developing innovative therapies such as drug or diet therapy, surgical approaches, and device development. Accurate epilepsy classification is essential in neurology, as it enables healthcare professionals to diagnose and treat patients with epilepsy effectively. Moreover, epilepsy classification aids in predicting prognoses, as different types of epilepsy have varying outcomes. e-learning platforms provide an accessible means for healthcare professionals to stay updated on the latest classification systems, guidelines, and treatment options for epilepsy.
Classification is always a dynamic process, aiming to gain new insights through research and a better understanding of these heterogeneous groups of diseases. Continuous evolution will further advance patient care in the future. Although a science-based classification is the ultimate goal, our current understanding is not yet advanced enough to achieve this. Thus, current recommendations are based on the latest scientific understanding combined with expert opinion, including consultation with epilepsy specialists and the worldwide epilepsy community (31-33). The ILAE, through its Classification and Terminology Commission, has developed a functional classification of seizures and epilepsy. Since the reorganization proposed in 2010 (6), further clarifications and feedback have been received from the community. One area requiring further explanation was the organization of various seizures. A seizure type classification task force was established in 2015 to provide recommendations for seizure classification (34). While our understanding of seizure networks is rapidly evolving, it is still not sufficiently used as a basis for seizure classification.
Research shows that designing and conducting an intervention should consider several aspects of active learning, including identifying learning needs, setting learning objectives, content relevance, outstanding learning methods and strategies, self-management strategies, and educational data transfer media. A system-level work environment significantly impacts the transfer of educational data to the learner. This includes practice opportunities, responsiveness, accurate monitoring, and other background factors that may influence the transfer of what is learned to the workplace. Designing and implementing an intervention focusing on active learning processes is critical in educational program design, and the factors mentioned should be considered in improving the effectiveness of educational data transfer (35).
e-learning has become an indispensable tool for medical residents in healthcare, providing numerous advantages that enrich their education and training experience. One of the primary benefits of e-learning is its flexibility. Medical residents typically have rigorous schedules, spending extended hours in hospitals and clinics. E-learning allows them to access educational resources at their own pace and convenience, seamlessly integrating learning into their hectic routines. This adaptability allows residents to harmonize their clinical duties with continuous education without compromising either aspect.
5.1. Strengths and Limitations
This study shows a tendency to replace traditional education with e-learning due to technological advances and changes in global educational approaches. Attention to infrastructure, proper education on the importance of e-learning, awareness of new developments, and consideration of learners' biological characteristics in course design will surely increase the success of e-learning in developing countries.
The results of this study should be confirmed by other studies with a larger sample size and longer review duration. It is also necessary to conduct more interventional and qualitative studies in this field. It is suggested that this approach be examined more broadly in Iran. Limitations of the study included intervening variables such as learner motivation, prior knowledge, and access to technology, as well as the small sample size. The study also faced challenges in establishing lasting learning effects.
If the present results are confirmed by studies with higher power, e-learning should be implemented as a permanent approach to increase the knowledge of specialized physicians in the country. It seems that the Kirkpatrick model is a suitable way to evaluate the effectiveness of in-service training in the field of health. Because the practical and clinical skills of specialist physicians are complex activities and the proper implementation of many of these skills ensures patient safety, it is recommended that learning and functional skills be continuously evaluated at regular intervals.
5.2. Conclusions
Nowadays, scientific human resources are considered the most important asset of an organization for gaining a competitive advantage. They are intangible assets that should be viewed as the key to improving the quality and efficiency of all organizational processes (36). Given that medical universities are major scientific institutions responsible for producing health knowledge and technology, training students and scientists, and providing services in the field of health, these organizations must continuously strive for excellence in promoting community health. However, upon examining the form and content of Iran’s educational system, it is clear that both general and higher education have several drawbacks (37).
e-learning enables learners to access content anytime, anywhere, while enriching it with a variety of resources and multimedia tools. It also allows teachers to review and update content instantly. However, the effectiveness of e-learning in educational institutions remains a subject of debate (38). The results of this study show that e-learning guidelines for the classification of epilepsy according to ILAE 2017 guidelines can be an effective tool in improving the clinical performance of pediatric neurologists.
