Abstract
Introduction:
Medical students should be trained about the fundamental skills of evidence- based medicine (EBM) during undergraduate medical education curriculum. This article introduces an innovative model for initiation and development of a flipped classroom for teaching EBM to sixth- year undergraduate medical students in Shiraz Medical School, affiliated to Shiraz University of Medical Sciences.Methods:
After conducting needs assessment and setting objectives for the program, blended leaning format and flipped classroom strategy were used. A student satisfaction measurement form was designed to evaluate the flipped classroom based on a 5-point Likert scale. The validity of the questionnaire was determined by experts, and the reliability of the questionnaire was 0.86 after a pilot study. We also used the questionnaires based on Berlin questionnaire to measure students’ knowledge, attitude, and practice about the EBM course in a pre- and posttest design. The validity and reliability of the questionnaire had been approved in our previous study. Also, the result of a summative examination after finishing the course was used as a measure of course goal achievement.Results:
A total of 280 medical students on their sixth year of study participated in the flipped classroom in 6 different rotations. A total of 266 (95%) students passed the course and only 14 (%5) failed. The results of posttest scores based on the EBM domains revealed that the best result was obtained in teaching clinical questions’ format (patient, intervention, comparison, and outcome). The lowest posttest scores were in the domain of biostatistics. The overall quality of flipped classroom was reported well by the students. The students stated that the ability to apply knowledge and skills in managing patients were developed in this course.Discussions:
In our experience, flipping classroom was a practical and essential activity to educate a large number of medical students about EBM. We acknowledge that the development of deep learning is instructors’ responsibility and that implementing blended learning and flipping classroom can facilitate students’ knowledge and skills about this important topic. We hope that other medical schools all over the world use the flipped classroom discussed in this article to redesign their classrooms in such a way that enables students to develop necessary skills and get involved in deep learning. This will improve the health of the society in the near future.Keywords
1. Introduction
Evidence-based medicine (EBM) is defined as ‘‘the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients’’ (1). EBM is an essential skill for medical doctors, and teaching EBM is included as an essential requirement in many medical schools all over the world (2).
Medical students should be trained about the fundamental skills of EBM during the undergraduate medical education curriculum (3). The undergraduate education in the 21st century project in the United States of America acknowledged practicing EBM as one of the 9 capacities to be combined into the medical school curriculum in the clinical years (4). The model of EBM and its educational implication have distributed quickly in recent years (5).The report of EBM education in most medical schools all over the world indicated that knowledge and satisfaction of students were increased after participating in these workshops (6-9).
In Iran, EBM education is considered in undergraduate medical curriculum in many medical schools. In Shiraz Medical School, the blended approach and flipped classroom model was used to teach EBM to medical students. Blended learning is defined as combining more than one method or technique for education (10). Currently, this method is defined as the systematic combination of online or multimedia with face-to-face teaching to increase communication among students, instructors, and resources (11).
This combination of online and face-to-face teaching incorporates well with the description of a flipped classroom. This concept developed in the 1990s and is defined as a subdivision of blended learning. Advances in active learning pedagogy, joined with improvements in instructional technology, have encouraged teachers to implement the flipped classroom (12, 13). Flipping the classroom creates a context that guarantees the students will obtain a personalized teaching tailored to their individual requirements. It can be used for teaching many concepts, including mathematical concepts. This method helps teachers to engage in a broad variety of learning styles and implement instructional strategies that improve problem-solving and critical thinking activities during the class period. Moreover, flipping the classroom encourages teachers to develop different learning experiences suitable for each student. This model is especially useful for busy and struggling students, it speaks the language of the students of the 21st century, and helps students of all capabilities to shine (13).
This article introduces an innovative model for needs assessment, initiation, and development of a flipped classroom model to teach EBM, moreover, it presents the mathematical concepts of this model to sixth- year undergraduate medical students in Shiraz Medical School, affiliated to Shiraz University of Medical Sciences.
2. Methods
2.1. Needs Assessment
Medical knowledge is rapidly changing. New diagnostics tests are therapies continuously being put on the market, and studies are always performed to review the effectiveness of existing treatment choices. EBM addresses these issues to keep clinicians up-to-date about new information. In our previous articles, we found that in spite of positive attitude toward EBM, different groups of residents were not familiar with EBM due to the lack of education about EBM in their undergraduate period (7, 14, 15).
In the present study, needs assessment was performed based on the following steps. First, the literature and available curriculums about teaching EBM to undergraduate medical students were reviewed. Then, expert opinion about introducing new courses or modifying existing ones were collected. In the third step, discussions with the medical education faculties and program directors was done during meetings to measure the need for announcing EBM in the medical students’ undergraduate curriculum and the method of teaching this important topic.
2.2. Defining Goals and Objectives
After reviewing the literature and consulting with experts, the goals and objectives of the course were designed. The main objectives were defined based on EBM processes as follows:
- Defining EBM
- Asking answerable clinical questions
- Searching for the best evidences
- Appraising the evidences
- Biostatistics
2.3. Using Blended Learning (Flipped Classroom Strategy)
Our course design was encouraged by a need to change the educational experiences of our students and facilitate active learning activities. The goals of our course design were to involve students through class activities and stimulate higher-order thinking using E-learning technologies and classroom teaching.
Before the class, we announced that the EBM course was a blended course and asked the students to study EBM principles. We arranged the class time with 4 active learning exercises. After a brief explanation about the concept of EBM, EBM questions in a PICO format were written as the first class activity. Formative assessment was done based on writing PICO for each scenario by students. The second activity was searching PICO questions in EBM sites based on the 5S Haynes Model. Another activity was the critical appraisal of a treatment article based on critical appraisal methods. In another section, the tutor provided explanation about relative risk, absolute risk reduction, number needed to treat and number needed to harm (8), and other treatment concepts. Then, the students solved problems related to the above- mentioned indicators. The goals were to provide students with fundamental concepts and create opportunities to have in-class activities about the above- mentioned concepts. After these activities, the students were asked to watch interactive CDs about critical appraisal of diagnosis studies, systematic reviews, and meta-analyses based on a standard offline format.
Appropriate formative and summative assessment was a central element of this course. Active involvement in group activities and shared activities was the criteria for giving bonus to students (about 5% of their final grade). Moreover, 3 graded quizzes (worth a total of 20% of their final grade) and 1 standard final examination based on in-class and offline content (worths 75%) evaluated the students’ understanding of the content and measured students’ capability to reach favorite learning outcomes.
We focused on 3 important basics in this flipped classroom activity: in-class student-centered activities, standard offline content, and appropriate formative assessment. The topics and mode of teaching are demonstrated in Table 1.
Topics Presented in the EBM Course
| Topic | Content | Teaching method |
|---|---|---|
| Introduction to EBM | The need for EBM with examples of EBM in clinical practice | In classroom |
| Writing an answerable question | The PICO components, students practicing writing questions using PICO from a selection of clinical scenarios. | In classroom |
| Searching the EBM databases | Students receive an interactive lecture on how to effectively search EBM sites. Students were asked to do a comprehensive search based on the previous section PICOs. | In classroom |
| Study designs | The following study designs were discussed | In classroom |
| - Randomized controlled trials | In classroom | |
| - Cohort studies | In classroom | |
| - Case-control studies | In classroom | |
| - Systematic reviews | In classroom | |
| Critical appraisal | Critical appraisal techniques based on the type of articles | |
| - Therapy | In classroom | |
| - Diagnosis | Interactive CDs | |
| - Prognosis | Interactive CDs | |
| - Systematic reviews | Interactive CDs | |
| - Meta-analysis | Interactive CDs | |
| Biostatistics | Confidence intervals & p-values | In classroom |
| Relative risk, absolute risk reduction, number needed to treat | In classroom | |
| - Forest plot | Interactive CDs | |
| - Sensitivity, specificity, positive and negative predictive values, and likelihood ratios | Interactive CDs |
2.4. Participants
A total of medical students on their sixth year of study participated in the flipped classroom in 6 different rotations. Of them, 172 (62%) were female and 108 (38%) were male. This course was an obligatory course for sixth- year medical students in Shiraz Medical School.
2.5. Assessment of the Flipped Classroom
We designed a student satisfaction measurement form to evaluate the flipped classroom based on a 5-point Likert scale. The validity of the questionnaire was determined by experts and the reliability of the questionnaire was 0.86 after a pilot study. We also used questionnaires based on Berlin questionnaire (6) to measure students’ knowledge, attitude, and practice about the EBM course in a pre- and posttest design. The validity and reliability of the questionnaire had been approved in our previous study (7). Moreover, the result of a summative examination after finishing the course was used as a measure of course goal achievement.
3. Results
Review of the literature confirmed the significance of teaching EBM to undergraduate medical students. Several comments were given by faculties about the necessity of teaching this important topic.
Review of other undergraduate curriculum all over the world revealed that EBM was one of the best new themes included in the revised curriculums. Discussions with our medical education faculties and program directors revealed that all of them emphasized the necessity of introducing EBM in the undergraduate medical curriculum.
A total of 280 sixth- year medical students participated in the flipped classroom in 6 different rotations. Of the students, 266 (95%) passed the course and only 14 (5%) failed. The percent of correct answers in each EBM domain in pre- and posttests are demonstrated in Table 2. As it is presented in Table 2, the best posttest result was obtained in teaching PICO and the lowest in biostatistics.
Mean and Standard Deviation of Students’ Scores on the Domains of EBM
| EBM Domain | Test | Mean ± SD (from 5) | P Value |
|---|---|---|---|
| Introduction to EBM | Pre | 1.98 ± 0.83 | < 0.001 |
| Post | 3.86 ± 1.23 | ||
| Formulating PICO questions | Pre | 1.81 ± 0.72 | < 0.001 |
| Post | 4.46 ± 0.831 | ||
| Effective searching strategy | Pre | 1.57 ± 0.47 | < 0.001 |
| Post | 3.96 ± 1.51 | ||
| Critical appraisal of evidences | Pre | 1.07 ± 0.63 | < 0.001 |
| Post | 3.93 ± 0.68 | ||
| Biostatistics | Pre | 1.72 ± 0.43 | < 0.001 |
| Post | 3.15 ± 1.096 |
All 280 undergraduate medical students (100%) responded to the satisfaction questionnaire (Table 3). The students reported good quality of flipped classroom in most domains.
Students’ Satisfaction about Flipped Classroom in EBM Course
| Course Evaluation Response | Number of Responding Students | Mean ± SD (from 5) |
|---|---|---|
| Instructors encouraged active students’ participation in class activities. | 276 | 04.41 ± 0.41 |
| Learning resources were useful. | 280 | 3.25 ± 0.36 |
| Teaching and learning strategies encouraged understanding of key concepts. | 277 | 4.62 ± 0.38 |
| Preparation before class is useful. | 269 | 2.31 ± 0.20 |
| Knowledge and skills gained in this course will be important for me in the future. | 275 | 3.53 ± 0.49 |
| The ability to apply knowledge and skills in managing patients was developed in this course. | 274 | 4.73 ± 0.67 |
| The CDs’ course content was interactive. | 280 | 3.26 ± 0.17 |
| My overall rating of course was good. | 280 | 4.31 ± 0.33 |
4. Discussion
EBM is a method that can strengthen critical and logical thinking and provide the opportunity for ideal clinical decision- making. Review of the literature showed the necessity of teaching EBM to undergraduate medical students to make them conscious about the models of working in clinics based on the best available and appraised evidences about diagnosis and treatment of diseases (6, 14, 16-19). A study on teaching EBM in UK medical schools suggested that EBM was a theme running through the whole curriculum, integrating the EBM into every part of the curriculum (8).
To our knowledge, this was the first study to evaluate the use of a blended EBM education using the flipped classroom model. We found that our blended learning and flipped classroom strategy significantly improved medical students’ EBM knowledge and skills. The students reported high satisfaction of the course, especially in the items such as teaching and learning strategies that encourage understanding of key concepts. The ability to apply knowledge and skills to manage patients and instructors encouraged active participation in students. The overall rating of the course was good. It seems that the characteristic of a blended module course encouraged students to participate more and improved their understanding.
The results of a study about comparing lecture and blended learning revealed that students’ satisfaction and cost efficiency in the blended learning model were more than lecture. The authors suggested that instructors should use blended learning to combine E-learning with theoretical teaching methods (20).
In a study on flipping the classroom, the authors concluded that this method improves deep learning and prepared students to address healthcare requirements of the 21st century (12). Another study about flipped classroom revealed that medical students had usually expressed appreciation for readily accessed online facilities as well as engaging small group in-class activities (21).
According to the results of the present study, only 5% of the students failed and the posttest results were significantly better than pretest results. Other studies showed the positive effects of EBM education on students’ knowledge (5, 7, 9). The results of posttest scores based on the EBM domains revealed that the best result is obtained in teaching PICO, which is due to the fact that clinical cases for EBM teaching are straightforward and clearly talk about the 4 constituents of the PICO questions. The results of the study on teaching EBM based on an integrative approach revealed the same results (22). The lowest posttest scores were in the domain of biostatistics; this may be due to unfamiliarity of medical students with these concepts. The same results were observed in our previous study (7, 14).
Research on EBM is one of the medical education research priorities in Eastern Mediterranean Region and Iran (23, 24). In Iran, the integration of medical schools with health system prepared a unique opportunity to use health care facilities to improve medical education (25). This opportunity will provide a good environment for integrated teaching of some concepts such as EBM. Teaching EBM to undergraduate medical students in a setting, in which they can directly use their EBM knowledge and skills in routine daily practice and have that training strengthened by instructors, who encourage the use of an EBM method in clinical practice, may extend their inclination to practice according to the basic principles of EBM in their lifelong professional activities (26, 27).
The strength of the present study was that this it was a good start for designing blended courses and flipped classroom activities based on an important topic like EBM for undergraduate medical students. One of the limitations of the present study was that we did not have a control group. Another limitation was that the formative and summative assessment of EBM was done soon after the course, so long-term outcomes were not obtained. Further assessment during clinical rotation would be helpful. Another limitation of the study was that the course was designed for the sixth- year medical students, which may not be a good representative of all medical students. Besides, the duration of the course was short and the pre-class activity was not fully structured. In future studies, a longer duration, especially for pre-class activities, would allow students to study more before the class, which would result in better attainment of the objectives.
4.1. Conclusions
Implementation of EBM courses for undergraduate medical students can be labeled as necessary. There is an agreement between all stakeholders about the necessity for a course on EBM. We commend that EBM be more considered as an important topic in the core curriculum of medical schools. In our experience, flipping classroom was a practical and essential activity to educate a large number of medical students about EBM. We acknowledge that the development of deep learning is instructors’ responsibility and implementing blended learning and flipping classroom can facilitate students’ knowledge and skills about this important topic. We hope that other medical schools all over the world use the flipped classroom discussed in this article to redesign their classrooms to enable students to develop necessary skills and get involved in deep learning. This will improve the health of the society in the near future.
Acknowledgements
References
-
1.
Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. BMJ. 1996;312(7023):71-2. [PubMed ID: 8555924].
-
2.
Dawes M, Summerskill W, Glasziou P, Cartabellotta A, Martin J, Hopayian K, et al. Sicily statement on evidence-based practice. BMC Med Educ. 2005;5(1):1. [PubMed ID: 15634359]. https://doi.org/10.1186/1472-6920-5-1.
-
3.
Glasziou P, Burls A, Gilbert R. Evidence based medicine and the medical curriculum. BMJ. 2008;337. a1253. [PubMed ID: 18815165]. https://doi.org/10.1136/bmj.a1253.
-
4.
O'Connell MT, Pascoe JM. Undergraduate medical education for the 21st century: leadership and teamwork. Fam Med. 2004;36 Suppl:S51-6. [PubMed ID: 14961403].
-
5.
Sanchez-Mendiola M, Kieffer-Escobar LF, Marin-Beltran S, Downing SM, Schwartz A. Teaching of evidence-based medicine to medical students in Mexico: a randomized controlled trial. BMC Med Educ. 2012;12:107. [PubMed ID: 23131115]. https://doi.org/10.1186/1472-6920-12-107.
-
6.
Fritsche L, Greenhalgh T, Falck-Ytter Y, Neumayer HH, Kunz R. Do short courses in evidence based medicine improve knowledge and skills? Validation of Berlin questionnaire and before and after study of courses in evidence based medicine. BMJ. 2002;325(7376):1338-41. [PubMed ID: 12468485].
-
7.
Mirzaei K, Zahmatkesh S, Amini M. Effect of evidence-based Medical Education on knowledge and ability to use and apply it among clinical students of Bushehr University of Medical Sciences: a controlled trial. Iran South Med J. 2016;19(3):398-410. https://doi.org/10.18869/acadpub.ismj.19.3.398.
-
8.
Meats E, Heneghan C, Crilly M, Glasziou P. Evidence-based medicine teaching in UK medical schools. Med Teach. 2009;31(4):332-7. [PubMed ID: 19404893]. https://doi.org/10.1080/01421590802572791.
-
9.
Ma X, Xu B, Liu Q, Zhang Y, Xiong H, Li Y. Effectiveness of evidence-based medicine training for undergraduate students at a Chinese Military Medical University: a self-controlled trial. BMC Med Educ. 2014;14:133. [PubMed ID: 24996537]. https://doi.org/10.1186/1472-6920-14-133.
-
10.
Sadeghi R, Sedaghat MM, Ahmadi FS. Comparison of the effect of lecture and blended teaching methods on students’ learning and satisfaction. J Adv Med Educ Profession. 2014;2(4):146.
-
11.
Garrison DR, Kanuka H. Blended learning: Uncovering its transformative potential in higher education. Internet High Educ. 2004;7(2):95-105. https://doi.org/10.1016/j.iheduc.2004.02.001.
-
12.
McLaughlin JE, Roth MT, Glatt DM, Gharkholonarehe N, Davidson CA, Griffin LM, et al. The flipped classroom: a course redesign to foster learning and engagement in a health professions school. Acad Med. 2014;89(2):236-43. [PubMed ID: 24270916]. https://doi.org/10.1097/ACM.0000000000000086.
-
13.
Bergmann J, Sams A. Flip your classroom: Reach every student in every class every day. International Society for Technology in Education; 2012.
-
14.
Karimian Z, Kojuri J, Sagheb MM, Mahboudi A, Saber M, Amini M, et al. Comparison of residents' approaches to clinical decisions before and after the implementation of Evidence Based Medicine course. J Adv Med Educ Prof. 2014;2(4):170-5. [PubMed ID: 25512942].
-
15.
Moghadami M. Internal medicine residents’ views and understanding of evidence based medicine. J Med Educ. 2009;8(1).
-
16.
Busari JO, Scherpbier AJ, van der Vleuten CP, Essed GG. The perceptions of attending doctors of the role of residents as teachers of undergraduate clinical students. Med Educ. 2003;37(3):241-7. [PubMed ID: 12603763].
-
17.
Heneghan C, Badenoch D. Evidence-based medicine toolkit. New Jerssy: BMJ Publishing Group Blackwell Publishing John Wiley & Sons; 2008.
-
18.
Nordenstrom J. Evidence-Based Medicine: In Sherlock Holmes' Footsteps. New Jerssy: John Wiley & Sons; 2008.
-
19.
Walczak J, Kaleta A, Gabrys E, Kloc K, Thangaratinam S, Barnfield G, et al. How are "teaching the teachers" courses in evidence based medicine evaluated? A systematic review. BMC Med Educ. 2010;10:64. [PubMed ID: 20920240]. https://doi.org/10.1186/1472-6920-10-64.
-
20.
Sadeghi R, Sedaghat MM, Sha Ahmadi F. Comparison of the effect of lecture and blended teaching methods on students' learning and satisfaction. J Adv Med Educ Prof. 2014;2(4):146-50. [PubMed ID: 25512938].
-
21.
Ramnanan CJ, Pound LD. Advances in medical education and practice: student perceptions of the flipped classroom. Adv Med Educ Pract. 2017;8:63-73. [PubMed ID: 28144171]. https://doi.org/10.2147/AMEP.S109037.
-
22.
Atwa H, Abdelaziz A. Evidence-based medicine (EBM) for undergraduate medical students: A six-step, integrative approach. Med Teach. 2017;39(sup1):S27-32. [PubMed ID: 28103721]. https://doi.org/10.1080/0142159X.2016.1254750.
-
23.
Nabeiei P, Amini M, Ghanavati S, Marhamati S. Research priorities in medical education at Shiraz University of Medical Sciences:categories and subcategories in the Iranian context. J Adv Med Educ Prof. 2016;4(1):26-32. [PubMed ID: 26793723].
-
24.
Amini M, Kojuri J, Lotfi F, Karimian Z, Abadi AS. Research priorities in medical education in the Eastern Mediterranean Region. East Mediterr Health J. 2012;18(7):687-92. [PubMed ID: 22891514].
-
25.
Salehi A, Harris N, Lotfi F, Hashemi N, Kojouri J, Amini M. Reform in medical and health sciences educational system: a Delphi study of faculty members' views at Shiraz University of Medical Sciences. East Mediterr Health J. 2014;20(3):151-61. [PubMed ID: 24950072].
-
26.
Ilic D, Forbes K. Undergraduate medical student perceptions and use of Evidence Based Medicine: a qualitative study. BMC Med Educ. 2010;10:58. [PubMed ID: 20718992]. https://doi.org/10.1186/1472-6920-10-58.
-
27.
Amini M, Kojuri J, Karimian Z, Lotfi F, Moghadami M, Dehghani MR, et al. Talents for future: Report of the second national medical science Olympiad in Islamic republic of Iran. Iran Red Crescent Med J. 2011;2011(6, Jun):377-81.