1. Background
Research in medical sciences is essential for identifying health problems and finding solutions to improve public health (1). In nursing, research is necessary to guide progress, support informed decisions, improve clinical practice, and enhance patient interaction (2). The nursing research course is challenging, and evaluations show teachers’ dissatisfaction as students struggle to apply their learning in practice (3).
For decades, lecture-based teaching has been a primary method for helping learners acquire new knowledge (4). However, many educators view this model as passive and teacher-centered (5). Technology is now a key part of education (6), offering teachers new opportunities for instruction (7). These technologies have enabled new methods, such as the flipped classroom (8). The flipped classroom is a dynamic, technology-based teaching method (9) that has gained attention from educators since 2004 (10). In this method, learners take primary responsibility (11), while the teacher acts as a facilitator, guiding and providing feedback (12, 13). In this method, learners receive the course content before class (14). The flipped classroom supports diverse learning styles, promotes active learning, interaction with the instructor, and a deeper understanding of the material (15, 16). Class time in this method focuses on thinking skills, in-depth topic exploration, and engaging learning experiences (17, 18). The flipped classroom has been shown to enhance classroom dynamics, motivation, deeper learning (19), and student outcomes (20-22).
The nursing research course aims to teach learners research design, but many find it boring and difficult (23); active methods like the flipped classroom can improve learning. Experts emphasize the importance of using the flipped classroom to teach the research course (24). Evaluating a training course is complex, and each method needs assessment; a comprehensive evaluation model is useful for measuring effectiveness and guiding teachers (25). This study used the Kirkpatrick model, a four-level framework suitable for evaluating educational programs (26), which posits that to evaluate educational courses, reaction, learning, behavior, and training results should be considered (27). This model is popular for its simplicity and ease of use, allowing even novice evaluators to design questions and indicators. It evaluates four levels: Reaction (learners’ response to the course), learning (skills and knowledge gained), behavior (changes in learners’ behavior), and results (achievement of organizational goals). While levels 1 and 2 are straightforward, levels 3 and 4 require long-term assessment, making them time-consuming (28). As societies progress, nursing education requires transformation. New, student-centered methods like the flipped classroom enhance clinical decision-making and foster motivated, creative graduates. Few studies have examined its use in nursing research courses, and results on student satisfaction remain mixed (28-30). For instance, some studies report higher satisfaction with lectures (31), while others note student frustration in the flipped classroom (32).
2. Objectives
This study aimed to evaluate the effectiveness of the flipped classroom in a master’s nursing research course by assessing both student learning outcomes and reactions, based on the first two levels of Kirkpatrick’s evaluation model.
3. Methods
3.1. Study Design
This was a quasi-experimental study with a pretest-posttest and a control group. The study was conducted on all first-semester master's-level nursing students enrolled in the research course at Birjand University of Medical Sciences. The aim was to evaluate the effectiveness of the flipped classroom teaching method in the nursing research course, based on the first two levels of Kirkpatrick’s evaluation model (reaction and learning).
3.2. Participants
The study population included all first-semester master’s nursing students enrolled in the ‘Nursing Research’ course (n = 24). Sampling was performed using a census approach. Participants were assigned to either the intervention group (n = 12) or the control group (n = 12) through cluster randomization. Specifically, out of the four nursing majors, two were randomly allocated to the intervention group and the other two to the control group using a lottery method. The inclusion criteria were as follows: Being a first-semester MSc nursing student, enrollment in the research course, no prior participation in research training workshops, and a willingness to attend all sessions. Exclusion criteria included an unwillingness to continue in the study and absence from any of the flipped classroom sessions.
3.3. Intervention
The intervention group engaged in three consecutive weekly sessions, each lasting approximately two hours, utilizing the flipped classroom methodology. Before each session, students were provided with pre-class learning materials prepared by the research team, which included educational videos, PowerPoint slides, and pertinent reading materials. These resources were meticulously aligned with the course syllabus and current academic references and were distributed to students in digital formats (such as CDs or electronic files) at least one week in advance (11, 14).
During the in-class sessions, active learning strategies were prioritized. Each session commenced with a brief instructor-led introductory lecture, followed by the completion of a six-item "study readiness form" to evaluate students' engagement with the pre-class material. Subsequently, students were organized into small groups of four to five members to engage in discussions and analyze pre-designed educational scenarios pertinent to the session's content. The instructor facilitated the group activities, addressed students' inquiries, and guided the discussion process to reinforce key learning points. The flipped classroom intervention was executed in seven structured stages.
In contrast, the control group received the same educational content through traditional lecture-based instruction, delivered by the same faculty member. No pre-class preparation or group-based activities were incorporated for the control group. To minimize the risk of cross-contamination between groups, the intervention and control classes were scheduled on different days and conducted in separate physical locations. Participants were also explicitly instructed not to share course materials or discuss class content with peers in the opposite group.
3.3.1. Scales
3.3.1.1. Reaction – Level 1 (Student Satisfaction)
Student reactions to the instructional method were assessed using a Researcher-developed Satisfaction Questionnaire. The instrument comprised 16 items categorized into three domains: Content (5 items), teaching method (7 items), and facilities (4 items). Each item was rated on a 5-point Likert scale ranging from 0 (very low) to 4 (very high), with a total score range of 0 to 64. Based on the total score, satisfaction levels were categorized as low (0 - 22), medium (23 - 45), and high (46 - 68), following expert judgment in nursing education and similar studies. The questionnaire was completed by the intervention group after each flipped classroom session. The content validity of the instrument was verified by a panel of nursing education experts. Reliability was confirmed through internal consistency analysis, yielding a Cronbach’s alpha of 0.85, indicating high reliability.
3.3.1.2. Learning – Level 2 (Knowledge Acquisition)
To evaluate student learning, a 20-item multiple-choice test was developed by the research team. The test encompassed key concepts in nursing research, such as research problem identification, literature review, research design, sampling methods, and data collection tools. The same test was administered as both a pretest and posttest to students in both groups. Each correct answer was awarded 1 point, with no penalty for incorrect answers, resulting in a total possible score ranging from 0 to 20. The internal consistency of the knowledge test was assessed using Cronbach’s alpha, which was calculated as 0.76, demonstrating acceptable reliability.
3.4. Data Collection
Following the acquisition of ethical approval and informed consent, baseline demographic data and pretest scores were collected. The educational intervention was conducted over three weeks as previously described. Satisfaction data were gathered after each session within the intervention group. Upon completion of the final session, a posttest knowledge assessment was administered to both groups.
3.5. Data Analysis
Data were analyzed utilizing SPSS version 24 (IBM, Chicago, IL, USA). Given the small sample size (n < 50), the Shapiro-Wilk test was employed to evaluate the normality of data distribution. The results confirmed a normal distribution (P > 0.05). Independent t-tests were conducted to compare mean scores between the two groups (e.g., age, pre/posttest scores). Paired t-tests were applied to assess within-group differences (pretest vs. posttest).
3.6. Ethical Consideration
Participants were provided with clear explanations regarding the study's objectives and methods, along with assurances of confidentiality and the freedom to withdraw at any stage. Written informed consent was obtained from all participants. The study received approval from the Ethics Committee of Birjand University of Medical Sciences under code IR.BUMS.REC.1397.370.
4. Results
The participants included 24 master's students, 12 of whom were assigned to the intervention group and 12 to the control group. The two groups showed no significant difference in age (P = 0.36) or sex distribution (P = 0.60). The mean age of the students in the control group was 28.09 ± 5.39 years, and in the intervention group, it was 30.3 ± 6.41 years. The mean duration since graduation from the previous level, meaning a bachelor's degree, was 6.5 ± 6.39 years in the control group and 2.08 ± 1.78 years in the intervention group. Among the 24 students who participated in the study, 8 were female, representing 33.3% of the total sample. Regarding previous experience with flipped classroom methods, 95.45% (21 students) reported no prior exposure.
The normality of the data was assessed using the Shapiro-Wilk test due to the sample size being less than 50. The results confirmed that the data were normally distributed (P > 0.05) for all main outcome variables. The independent t-test showed no significant difference in pre-test learning scores between groups (P = 0.53). The increase in post-test scores within the intervention group compared to its pre-test was highly significant (paired t-test: t = -10.5, df = 11, P < 0.001). In addition, the mean post-test score in the intervention group (12.5 ± 1.95) was significantly higher than that in the control group (7.66 ± 3.11; independent t-test: t = 4.24, df = 22, P < 0.001, Table 1).
Table 1.The Mean Learning Score of the Participating Students, Before and After the Intervention, in the Two Experimental and Control Groups a
| Learning level | Intervention Group | Control Group | Between-Group Results b |
|---|---|---|---|
| Before intervention | 4.89 ± 1.75 | 5.50 ± 2.02 | t = -0.85, df = 22, P = 0.53 |
| After intervention | 12.5 ± 1.95 | 7.66 ± 3.11 | t = 4.24, df = 22, P = 0.15 |
| The difference before and after the intervention | 7.70 ± 2.31 | 2.16 ± 2.28 | t = 5.61, df = 21.22, P = 0.001 |
| Within-group results | t = -10.5, df = 11, P < 0.001 c | t = -3.27, df = 11, P = 0.007 c | - |
a Values are expressed as mean ± SD.
b Independent t-test (between-group comparison).
c Paired t-test (within-group comparison).
In terms of the reaction score, the mean reaction score after the intervention increased from the first session to the third session (from 48.80 to 51.20). Overall, the mean reaction score of students, considering the three levels of the method (0 - 22: Low, 23 - 45: Medium, and 46 - 68: High), was at a high level (Table 2).
| Reaction Score | Intervention Group |
|---|---|
| First session | |
| Content | 14.9 ± 3.38 |
| Teaching method | 19.9 ± 3.63 |
| Facilities | 11.3 ± 3.56 |
| Total satisfaction | 3.1 ± 0.73 |
| Total | 48.80 ± 7.92 |
| Second session | |
| Content | 15.5 ± 4.4 |
| Teaching method | 20.4 ± 4.78 |
| Facilities | 11.8 ± 2.48 |
| Total satisfaction | 3.4 ± 0.69 |
| Total | 51.10 ± 9.65 |
| Third session | |
| Content | 15.5 ± 3.3 |
| Teaching method | 20. 9 ± 4.45 |
| Facilities | 11.7 ± 2.71 |
| Total satisfaction | 3.1 ± 0.87 |
| Total | 51.2 ± 10.09 |
a Values are expressed as mean ± SD.
b Average of session totals: 50.37 (descriptive only, no SD calculated).
5. Discussion
This study used the flipped classroom to teach the nursing research course. The method significantly improved students' learning outcomes and post-test scores compared to lectures. Students in the flipped classroom also reported high satisfaction with the method and content. Studies report mixed results on the flipped classroom, so comparing our findings with prior research situates them in the broader context. Shwu-Ru et al. found that 87% of nursing students were satisfied with the flipped classroom and learned significantly (23). Similar to Shwu-Ru et al., this study found that the flipped classroom improved learning and satisfaction, with students valuing the content and pre-class videos. Differences include a smaller sample size and focus on Kirkpatrick’s first two levels (reaction and learning), but both studies support the method’s usefulness in the research course (23).
El-Banna et al. compared flipped and traditional classrooms in pharmacology. Flipped classroom students scored higher on the first exam, but satisfaction was greater with lectures, with no difference in the second exam. Both quantitative and qualitative results showed higher student satisfaction with lecture-based teaching (P < 0.001) (31). This study aligns with ours; however, it used a mixed-methods design with qualitative data from focus groups. In their study, groups experienced both methods in a crossover design, while our study applied the flipped classroom only to the intervention group and lectures to the control group (31). El-Banna et al.’s findings on student satisfaction differed from ours, likely due to differences in sampling and course nature (31).
Harrington et al. found no significant difference in learning outcomes between lectures and flipped classrooms (P = 0.09) (33), differing from our results, likely due to course type, student level, and evaluation method. Similarly, Dabney and Mitchell reported student frustration with flipped classroom management and lower overall satisfaction (32). In contrast, our students reported high satisfaction, likely due to differences in implementation, course design, or instructor facilitation. Overall, flipped classroom success may depend on course content, instructional design, student readiness, and institutional support.
5.1. Conclusions
This study found that the flipped classroom significantly improves learning outcomes compared to traditional lectures. The intervention group’s learning scores increased significantly, showing the method’s effectiveness in boosting engagement and knowledge retention. Furthermore, the high reaction scores indicate students had a positive perception of the flipped classroom. These results support integrating flipped classrooms to enhance active learning and academic performance.
5.2. Strengths
A key strength is applying the flipped classroom to the challenging nursing research course. The study contributes to the limited research on active learning in nursing research courses. Additionally, the use of Kirkpatrick’s model for evaluation provided a structured and systematic approach to assessing both student reactions and learning outcomes. The study also benefited from a real-world classroom implementation, allowing the intervention to be tested under practical teaching conditions. Furthermore, by including all eligible students in the research course during the semester, the study minimized selection bias.
5.3. Limitations
However, this study also has several limitations. First, due to time constraints, only the first two levels of Kirkpatrick’s model (reaction and learning) were evaluated; behavior and results at the organizational level were not assessed. Second, the sample size was relatively small, as it was limited to all eligible students enrolled in the nursing research course during the semester, which may affect the generalizability of the findings. Furthermore, the duration of the flipped classroom intervention was short (three sessions), which may not fully capture its long-term effects.
