Abstract
Background:
During the COVID-19 pandemic, educational programs have increasingly relied on virtual methods.Objectives:
In this study, we compared the effects of in-person and virtual electroencephalography (EEG) training on the knowledge of pediatric and adult neurology residents.Methods:
The study participants consisted of 30 pediatric and adult neurology residents who were recruited nationwide via a virtual network. They were randomly divided into two groups for in-person and virtual education, respectively. Prior to the first workshop session, participants completed a pre-test comprising 19 knowledge-related questions. Two interactive training sessions focusing on benign variants in EEG were conducted, including two in-person workshops for eight residents and two online workshops for 22 residents. Each session lasted one hour, with one held every two weeks. After the second session, participants were asked to complete a post-test consisting of 24 questions (identical to the pre-test) and five workshop satisfaction-related questions.Results:
Overall, the mean exam score after both the in-person and virtual workshops was significantly higher than the pretest score across all educational levels. The mean score differences in knowledge between the in-person and virtual groups after the workshop were similar, with no statistically significant difference. Approximately 90% of the participants believed that the workshop would be beneficial for their daily practice.Conclusions:
Given the absence of a significant difference between in-person and virtual training methods in improving participants' knowledge and satisfaction with both approaches, we recommend integrating virtual EEG training into the educational protocol for pediatric and adult neurology residents. Nonetheless, larger sample size studies are required to further validate these findings.Keywords
1. Background
The Clinical Neurophysiology Fellowship attracts numerous pediatric neurologists with a keen interest in epilepsy, who often specialize in EEG. Nevertheless, it is essential that all pediatric neurologists treating epilepsy patients possess a fundamental understanding of pediatric EEG. Key skills for pediatric neurologists in this field include recognizing age-related patterns, interpreting EEG during sleep and wakefulness, identifying the effects of photic stimulation and hyperventilation, diagnosing artifacts, identifying EEG patterns associated with childhood epilepsy, and recognizing the characteristics of status epilepticus and other seizure types (1).
During the COVID-19 pandemic, educational programs have increasingly relied on virtual methods (2). However, there have been limited published studies investigating the outcomes and impact of such training (3).
A recent survey of epilepsy fellowship programs in the United States revealed that program directors estimated a moderate-to-severe impact of the COVID-19 pandemic on education at 30%, while fellows estimated it at 49%. Additionally, 20% of program directors and colleagues expressed that further training was necessary before graduation (1).
In a study conducted by Ochoa and Naritoku in 2012, 20 neurologists were enrolled. They completed a pre-test consisting of 40 questions, then participated in a virtual course on EEG basics and 40 clinical EEGs, and finally took a 40-item post-test. The results showed that all participants significantly improved their scores in the post-test compared to the pre-test. The mean score in the pre-test was 61.7%, while the mean post-test score was 87.8%. The study concluded that virtual EEG training could effectively enhance EEG knowledge among neurologists (4).
In a study by Bensalem-Owen et al., which included 10 anesthesia residents, EEG training was delivered via a podcast instead of a traditional lecture. Prior to the training course, learners were administered a pre-test consisting of 25 questions. They were then trained using both traditional and web-based lecture methods, including an explanation of 10 clinical EEG cases, before completing a post-test consisting of 25 questions. The results indicated no significant difference in educational effectiveness between the traditional and web-based lecture methods (5).
In a 2017 study by Moeller, assistants in neurological rotation were provided with a curriculum consisting of 10 short (6 - 17 minutes) EEG training videos, which they were asked to review and read in sessions. The educational objectives included technical understanding of EEG and its relationship to EEG interpretation, understanding of simple EEG findings, including sleep-wake patterns and common abnormal findings, and using trained knowledge to interpret routine EEGs. An evaluation was conducted at the end of the year, and almost all of the participants believed that this curriculum was a useful part of the rotation and helped them to understand complex concepts (6).
One of the most important issues in reading an electroencephalography is identifying normal variants (benign variants). Failure to correctly diagnose these variants may lead neurologists to mistake them for seizure waves, which could result in inappropriate administration of anti-seizure medication, increased side effects, and unnecessary medical expenses.
2. Objectives
In this study, we investigated the impact of virtual and in-person EEG training on the knowledge of pediatric and adult neurology residents during the COVID-19 pandemic.
3. Methods
Participants in this study consisted of 30 pediatric and adult neurology residents who were recruited from across Iran through virtual networks. They were then divided into two groups: one received in-person education, while the other received virtual education. All participants were provided with an informed consent form to review and sign prior to participating in the study.
The pre-test and post-test multiple-choice questions regarding benign variants in EEG were reviewed and validated by three neurophysiology experts to ensure content validity and relevance. Clarity was considered acceptable if the total agreement was at least 70%, as suggested by Davis. In our study, the inter-rater agreement (IRA) for clarity was 100% using a less conservative approach. Relevancy was also considered acceptable if the total agreement was at least 70%, with the IRA for relevancy in our study is 100% using a less conservative approach.
Then, for the evaluation of reliability, it was piloted by ten pediatric and adult neurology residents. Reliability was evaluated by test-retest, and the questionnaire was filled out by ten pediatric neurology residents twice at a 3-week interval. The acceptable intraclass correlation coefficient (ICC) was considered to be 0.7, and questions with indexes below 0.7 were omitted from the questionnaire.
Before the first workshop session, participants completed an online pre-test consisting of 19 knowledge-related questions. Then, interactive training sessions about benign variants in EEG were held, including two in-person workshops for eight residents of our hospital and two online workshops for 22 residents of other hospitals. Each session lasted one hour, with one after two weeks of another. After the second session, participants completed an online post-test consisting of 24 questions (identical to the pretest with five satisfaction-related questions).
The content of the workshops included posterior slow waves of youth, Lambda waves, positive occipital sharp transients of sleep, Mu waves, breach rhythm, wicket rhythm, phantom waves, hypnogogic hypersynchrony, psychomotor variants, and benign epileptiform transients of sleep. The questions in the pre-and post-test were related to the content of the workshops with multiple choice questions, and some included epochs related to different benign variants. The virtual and in-person workshops were designed to be interactive, and participants could raise their hands to answer questions or ask any questions.
Because our curriculum is different from other countries, with three years of pediatrics and two years of pediatric neurology fellowship, we considered the first-year fellowship as a fourth-year pediatric neurology resident and the second-year fellowship as a fifth-year pediatric neurology resident for the common terminology.
Mean and standard deviation were used to describe quantitative variables, and for qualitative variables, frequency and percentage were used. Paired t-test was used to evaluate the score before and after in the dependent groups, and an independent sample t-test was used to evaluate the score in two independent groups. The Likert scale was used to assess satisfaction with the workshop.
4. Results
The most commonly used device for participating in the pretest and post-test was a cellphone, with 83% and 86% usage, respectively. Out of 30 participants, four were male, and 26 were female, with a mean age of 37.1 ± 4.9 years. Twenty-one participants were pediatric neurology residents, and 9 were adult neurology residents. Only seven residents had previously attended similar workshops.
On average, the post-workshop exam scores were higher than the pretest scores for all educational levels, and this difference was statistically significant. The average score differences in knowledge for the in-person and virtual groups were similar, and there was no statistically significant difference between the two. Further details of the mean score of knowledge in different educational levels are shown in Table 1. The mean scores for knowledge before and after the virtual and in-person workshops are presented in Table 2.
Average Knowledge Scores in Different Educational Levels Before and After the Workshop
| No. | Mean ± SD | Std. Error Mean | P-Value | |
|---|---|---|---|---|
| First and second year ANR | ||||
| Pre-test | 3 | 5.3333 ± 4.16333 | 2.40370 | 0.02 |
| Post-test | 3 | 15.6667 ± 2.30940 | 1.33333 | |
| Third and fourth-year ANR | ||||
| Pre-test | 6 | 8.1667 ± 2.48328 | 1.01379 | 0.02 |
| Post-test | 6 | 14.1667 ± 2.78687 | 1.13774 | |
| Fourth year PNR | ||||
| Pre-test | 7 | 7.7143 ± 2.13809 | 0.80812 | 0.01 |
| Post-test | 7 | 13.4286 ± 4.15761 | 1.57143 | |
| Fifth year PNR | ||||
| Pre-test | 14 | 10.4286 ± 2.87467 | 0.76829 | 0.0001 |
| Post-test | 14 | 15.4286 ± 1.91007 | 0.51049 | |
| All | ||||
| Pre-test | 30 | 8.8333 ± 3.13031 | 0.57151 | |
| Post-test | 30 | 14.7333 ± 2.77841 | 0.50727 | 0.0001 |
Average Knowledge Scores Before and After the Virtual and In-person Workshops a
| Workshop | No. | Mean ± SD | Std. Error Mean | P-Value |
|---|---|---|---|---|
| Virtual | 0.0001 | |||
| Pretest | 22 | 8.1364 ± 2.86681 | 0.61121 | |
| Posttest | 22 | 14.8182 ± 2.46183 | 0.52486 | |
| In-person | 0.01 | |||
| Pretest | 8 | 10.7500 ± 3.19598 | 1.12995 | |
| Posttest | 8 | 14.5000 ± 3.70328 | 1.30931 |
The level of satisfaction with the workshop in terms of timing and comprehensiveness was very high at 50%. Satisfaction with the platform, whether in-person or virtual, was also high at 46.7%. Approximately 90% of the participants believed that this workshop would be effective in their daily practice.
5. Discussion
One of the most important issues in reading electroencephalograms is identifying normal variants or benign variants. Failure to diagnose these correctly may lead neurologists to mistake them for seizure waves, which can result in inappropriate administration of anti-seizure medication, increased side effects, and higher medical expenses. Due to the COVID-19 pandemic, in-person educational programs have been restricted, and most educational workshops are now held virtually.
At one hospital in Iran during the pandemic, the number of educational classes held through virtual spaces increased compared to one month prior to the outbreak. Telemedicine, including online patient follow-up using the telephone or internet, was also implemented. Changing from in-person practice to virtual learning and telemedicine was a practical and useful model that was recommended for other centers during the pandemic crisis (7).
In our study, we compared in-person versus virtual training platforms for benign variants in EEG. Our results were consistent with a study by Ochoa and Naritoku in which the average score significantly improved after virtual EEG training. We also compared the improvement of knowledge in the virtual group with the in-person group (4). Our findings were similar to those of a study by Bensalem-Owen et al. that involved 10 anesthesia residents, which concluded that there was no significant difference in educational effectiveness between traditional and web-based lecture methods (5).
A study by Hartzler et al. involved a cluster-randomized hybrid type 3 trial of contingency management implementation in opioid treatment programs (OTPs) to compare the cost-effectiveness of virtual versus in-person training. The results showed greater utility, lower expenses, and higher cost-effectiveness of virtual training (8). In our study, virtual workshops were more cost-effective, as the improvement of knowledge in virtual and in-person workshops was the same, but the expenses for in-person workshops were higher.
In a study by Yadala et al., 11 neurology residents underwent online and interactive EEG training for eight weeks. The results showed that the scores of the residents in the post-test were significantly higher than the pre-test, and 81.8% strongly agreed that virtual EEG training is an efficient method for training residents with easy access. Another result was increased confidence in residents due to the possibility of interactive discussions (9). In our study, virtual and in-person workshops were designed to be interactive, and participants could raise their hands to answer questions or ask questions.
One limitation of our study was the allocation of residents from our hospital to the in-person groups because of easier accessibility. Another limitation was our small sample size due to the COVID-19 pandemic, which restricted us from setting up a workshop with a larger sample size. We recommend future studies in this setting with a larger sample size to improve generalizability.
5.1. Conclusions
Based on the comparable efficacy and participant satisfaction of in-person and virtual training methods in improving EEG knowledge, it is recommended to incorporate virtual EEG training into the educational protocol for pediatric and adult neurology residents. However, future studies with larger sample sizes are recommended.
Acknowledgements
References
-
1.
Morse RP, Holmes GL. Education of a child neurologist: epilepsy and electroencephalography. Semin Pediatr Neurol. 2011;18(2):74-8. [PubMed ID: 22036483]. https://doi.org/10.1016/j.spen.2011.05.002.
-
2.
Cross JH, Kwon CS, Asadi-Pooya AA, Balagura G, Gomez-Iglesias P, Guekht A, et al. Epilepsy care during the COVID-19 pandemic. Epilepsia. 2021;62(10):2322-32. [PubMed ID: 34428314]. [PubMed Central ID: PMC8652685]. https://doi.org/10.1111/epi.17045.
-
3.
Van Winssen C, Kyriakopoulos P, El Sadi F, Rezazadeh A, Fantaneanu T. Integrating virtual EEG sessions into fellowship programs during the COVID-19 pandemic. Clin Neurophysiol Pract. 2022;7:96-7. [PubMed ID: 35281721]. [PubMed Central ID: PMC8904309]. https://doi.org/10.1016/j.cnp.2022.03.001.
-
4.
Ochoa J, Naritoku DK. Using a virtual training program to train community neurologist on EEG reading skills. Teach Learn Med. 2012;24(1):26-8. [PubMed ID: 22250932]. https://doi.org/10.1080/10401334.2012.641483.
-
5.
Bensalem-Owen M, Chau DF, Sardam SC, Fahy BG. Education research: evaluating the use of podcasting for residents during EEG instruction: a pilot study. Neurology. 2011;77(8):e42-4. [PubMed ID: 21860003]. https://doi.org/10.1212/WNL.0b013e31822b0017.
-
6.
Moeller JJ, Farooque P, Leydon G, Dominguez M, Schwartz ML, Sadler RM. A Video-Based Introductory EEG Curriculum for Neurology Residents and Other EEG Learners. MedEdPORTAL. 2017;13:10570. [PubMed ID: 30800772]. [PubMed Central ID: PMC6342517]. https://doi.org/10.15766/mep_2374-8265.10570.
-
7.
Zeinali M, Almasi-Doghaee M, Haghi-Ashtiani B. Facing COVID-19, Jumping From In-Person Training To Virtual Learning: A Review on Educational and Clinical Activities in a Neurology Department. Basic Clin Neurosci. 2020;11(2):151-4. [PubMed ID: 32855773]. [PubMed Central ID: PMC7368106]. https://doi.org/10.32598/bcn.11.covid19.910.2.
-
8.
Hartzler B, Hinde J, Lang S, Correia N, Yermash J, Yap K, et al. Virtual Training Is More Cost-Effective Than In-Person Training for Preparing Staff to Implement Contingency Management. J Technol Behav Sci. 2022:1-10. [PubMed ID: 36246531]. [PubMed Central ID: PMC9553630]. https://doi.org/10.1007/s41347-022-00283-1.
-
9.
Yadala S, Nalleballe K, Sharma R, Lotia M, Kapoor N, Veerapaneni KD, et al. Resident Education During COVID-19 Pandemic: Effectiveness of Virtual Electroencephalogram Learning. Cureus. 2020;12(10). e11094. [PubMed ID: 33110712]. [PubMed Central ID: PMC7581218]. https://doi.org/10.7759/cureus.11094.