1. Background
Quality feedback is essential for medical education, especially in traditional apprenticeship models. Jack Ende defined feedback as “information describing a learner’s performance in a given activity that is intended to guide or shape their future performance” (1). Effective feedback drives development, while poor feedback leads to stagnation (2). The best feedback is a cycle of specific and timely commentary comparing performance and expectation, given with an interest to improve the learner’s skillset (2).
Despite its value, feedback is inconsistently delivered across medical education. Much of it is vague and unhelpful, lacking the specificity needed to improve performance (3, 4). Literature frequently notes medical student feedback as “vague” or “non-constructive”. While specific feedback yields a significant improvement in performance, vague comments have no such effect (5). One study reported only 22% of all feedback was rated to be “high quality” (6, 7). Additionally, the traditional top-down feedback model limits relevance to learners as educators dictate the content and structure with often little influence from the learner (3, 8-11).
There is growing recognition of the importance of an “Educational Alliance” encouraging students and educators to actively participate in the feedback relationship. This fosters a two-way dynamic to enhance learning and maximize growth potential (12, 13). Additionally, such student-initiated feedback conversations have been shown to yield more specific and timely discussions with a better connection to the learner’s objective performance (7, 8, 14).
Fields in medicine frequently have unique challenges to successful feedback. In neurology, numerous factors may influence feedback infrastructures. The detailed, time-intensive nature of a neurologic assessment limits time for feedback while making it more essential to develop the bedside skills required in clinical education (15). Student anxiety about neurology, or “neurophobia”, can increase the need for feedback while making it harder to provide (16, 17). Effective feedback has been associated with greater student interest in neurology, performance on standardized tests, and learner satisfaction (18).
2. Objectives
We studied feedback quality in the neurology clerkship to better understand the core components of effective medical student feedback.
3. Methods
We performed a mixed-methods analysis of the student clerkship feedback infrastructure via a two-phased approach. Phase 1 included a quantitative and qualitative assessment of a novel self-reflective feedback form. Phase 2 utilized qualitative analysis of focus group discussions among key stakeholders in medical student feedback.
3.1. Phase 1: Redesigning the Traditional Feedback System
With the original feedback system utilized by the neurology clerkship at Mount Sinai, students requested a formative feedback form from a supervising resident or faculty member at their clerkship’s midpoint. The form asked for general information on student performance and suggested areas for improvement. Forms were collected and reviewed with the clerkship director. We introduced a novel learner-driven feedback form to encourage reflection and focused dialogue with educators. This exercise asked students to identify personal strengths and areas for improvement, guided by AAMC Core Entrustable Professional Activities, which were presented to educators to guide feedback conversations (19, 20). In April 2022, this replaced the prior generic faculty-completed form available to students for use in the neurology clerkship.
Students were asked to complete an anonymous survey via RedCap to evaluate their mid-clerkship feedback encounter. Using a 1 - 10 Likert scale, they rated feedback quality, usefulness, and impact, including whether it involved active discussion or self-assessment. Surveys distinguished the educator’s level of training and asked for narrative comments (Appendix 1 in Supplementary File). Surveys were distributed by an investigator not involved in grading, within five days of clerkship completion. Data was collected from August 2021 to November 2022, spanning periods before and after the form’s implementation. All students who completed their neurology clerkship during the data collection period were included and sent an invitation to complete the survey. There were no exclusion criteria applied within this group.
Pre- and post-implementation survey data were compared. Likert scale scores were compared using chi-squared tests for statistical significance. Narrative assessment was analyzed thematically. Data was stored electronically in a secure and anonymous database via RedCap with the permission of participants.
3.2. Phase 2: Focus Group Investigation of Medical Student Feedback
Following survey analysis, three 30-minute focus groups (students, residents, and attendings) were conducted between May 2023 and April 2024, using a semi-structured focus-group script, for qualitative investigation and deeper analysis of the ideal neurology feedback environment (Appendix 2 in Supplementary File). Focus group participants were selected by the study team based on participant background, interests, involvement with medical education, and level of training to prioritize diverse and informative viewpoints. The student focus group was held in person, while others were virtual for the scheduling convenience of participants. The lead investigator, familiar with the clerkship but with no role in its leadership, moderated the sessions. Sessions were recorded with consent.
All focus groups asked semi-structured questions of participants to provide their insight into the medical student feedback process in neurology and to respond to findings from the first phase of this project. Participants were also asked to comment on challenges or factors specific to feedback in the field of neurology. All data and recordings were stored securely and anonymously with the permission of participants. Audio recordings were then transcribed verbatim. Transcripts were coded line-by-line on a thematic basis. First-level descriptive coding was followed by second-level analytic coding (Appendix 3 in Supplementary File). Analysis was then conducted for thematic relevance by the study authors. Intervention and quantitative methods in this study were approved by the Icahn School of Medicine Institutional Review Board prior to the initiation of data collection.
4. Results
4.1. Phase 1: Redesigning the Traditional Feedback System
The survey was sent to a total of 286 students, with 174 students prior to form implementation and 112 students after implementation. Twenty-nine students completed the pre-implementation survey (response rate 16.6%), and seventeen completed the post-implementation survey (response rate 15.1%). Survey validity was acceptable with a Cronbach's Alpha value of 0.74. Among all forty-six students, only 22% (n = 14) received feedback from an attending, whereas 80% (n = 37) reported feedback from residents. Twenty-eight students received feedback only from residents, 8 received feedback only from attendings, and 6 received feedback from both residents and attendings. Five students received feedback from both a Junior and Senior resident.
Feedback was perceived similarly pre- and post-implementation and described as “useful and actionable” (86% pre vs. 71% post, P = 0.5) and an “active conversation” (86% vs. 77%, P = 0.5). Regardless of format, students felt feedback provided an opportunity for self-assessment (72% vs. 65%, P = 0.3), targeted areas for improvement (83% vs. 53%, P = 0.4), and was constructive and helpful (97% vs. 94%, P = 0.6) with both formats of feedback (Figure 1).
Figure 1.
Student impressions of attending vs. resident feedback, regardless of form utilization, on a Likert scale of 0 to 10
Students rated the quality of feedback received from residents higher than that from attendings (8.0 vs. 6.1 on a scale of 10, P = 0.004). Additionally, feedback obtained from residents was felt to be more applicable to yield substantive change in performance than feedback from attendings (7.6 vs. 5.9 out of 10, P = 0.02) (Table 1).
| Variables | Before (Agree) | After (Agree) | P-Value |
|---|---|---|---|
| Feedback was useful and actionable. | 86 | 71 | 0.5 |
| Feedback was given in an active conversation. | 86 | 77 | 0.5 |
| Feedback allowed for self-assessment. | 72 | 65 | 0.3 |
| Feedback targeted areas for improvement. | 83 | 53 | 0.4 |
| Feedback was constructive and helpful. | 97 | 94 | 0.6 |
a Values are expressed as percentage.
b Percent of students who self-reported to “Agree” or “Strongly Agree” with each statement regarding feedback received before and after implementation of the self-reflective form.
Qualitative comments were similar pre- and post-implementation. Regardless of format, students felt there was not enough time allotted to feedback and that it was given in haste. Additionally, students felt feedback lacked specificity and was impersonal. While students “enjoyed the ability to self-reflect” with the new form, they felt it was “intimidating” to ask attendings for feedback “given their busy schedules”. Students also found “people uncomfortable giving [feedback on what to work on more]”.
4.2. Phase 2: Focus Group Investigation of Medical Student Feedback
Focus groups included students entering various specialties, residents across training levels with varied subspecialty interests, and faculty from different neurology subspecialties. Faculty backgrounds included varied subspecialties as well as some clerkship and residency leadership involvement, offering diverse perspectives. All focus group participants agreed that current feedback systems are inadequate (Table 2). They noted vague, nonspecific comments as barriers to growth. Three key themes emerged: The need for strong educator-learner relationships, dedicated time for feedback, and clearer expectations for both parties.
Table 2.Medical Student Feedback Systems are Suboptimal a
| Quotes | Focus Group Source |
|---|---|
| “[The feedback is] just ‘ok good job’. The opportunities to be observed are sometimes too few.” | Student |
| “I don’t think I’ve gotten any feedback from attendings. We don’t know who writes our formal evaluation.” | Student |
| “Most feedback I have gotten...has been positive without anything constructive.” | Student |
| “I don’t think it goes well. It is not very structured. We don’t know how to give feedback well.” | Resident |
| “[Feedback] is not very structured. We give very basic feedback we don’t dive into things.” | Resident |
a Focus group quotations regarding suboptimal nature or medical student feedback systems.
4.2.1. The Crucial Relationship Between Learner and Educator for Optimal Feedback
All participants recognized the importance of an educational alliance between learner and educator to allow for effective feedback (Table 3) (11). Not surprisingly, students expressed that the time spent with residents facilitated closer relationships than with attendings. Participants acknowledged the proximity in professional development of students to residents, as compared to attendings, as a major contributor to feedback quality. This relationship was emphasized as a likely factor in higher perceived satisfaction with feedback from residents compared to attendings (Table 3).
Table 3.The Crucial Relationship Between Learner and Educator a
| Quotes | Focus Group Source |
|---|---|
| Quotes on the importance of an educational alliance | |
| “I think a lot of it boils down to just time you spend with someone. When you spend time with someone, you get more invested. If you’re more invested, you’re more willing to give better feedback.” | Student |
| “If you don’t have a relationship with the person, you can’t take [their] feedback seriously. If [the person giving feedback] doesn’t know you, how is there any validity.” | Resident |
| “[It is] incredibly important to have a relationship to student and often give the best feedback to those I can have a longitudinal relationship with. I wish there were ways to establish meaningful relationship on the wards.” | Attending |
| Quotes on student relationships with residents as compared to attendings | |
| “One thing I found helpful on this rotation; I went to my outpatient clinic day with a resident I had worked with. I thought it was helpful to be with that person in a different setting. Though it was a different attending, but the resident got a better sense of how I work.” | Student |
| “I’m curious how the different services give feedback but again it boils down to time. I spend more time with the residents. You go see the patients together, so it gives a good opportunity to see the exam and the history and to give that feedback.” | Student |
| “We don’t spend as much time with the attendings. There’s a collaborative environment among residents and since they are closer to our chapter, they feel more inclined to give feedback. You can tell there is more investment from [residents]. They are more able to give day to day impressions of how we do. With attendings it seems to come down to more of their personality and if they want to teach and connect with the students.” | Student |
| “Residents have more facetime with the students. You are together all day. There’s more empathy there, you’ve been there more recently so you’re more attuned to what they want to hear.” | Resident |
| “Residents were more recently students. [They have a] clearer sense of what the students are looking for and what their skill set is.” | Attending |
| Quotes on barriers to building an effective educational alliance | |
| “Residents need feedback more than us as they’re closer to actual clinical practice. But attendings need to remember that we still must build a foundation to become doctors.” | Student |
| “Once you implement [feedback], there is no further development or progression, it is just ‘ok good job’. The opportunities to be observed are sometimes too few.” | Student |
| “Feedback is perceived to be better by residents because we are closer in age and more like [the students]. We are less intimidating.” | Resident |
| “The way we give feedback is silly. It’s at the end of the block. The students get feedback and then move on to other rotations, or we move on to other services. So how [can we] see them implement it?” | Resident |
| “One barrier [to effective feedback] was turnover with the faculty and residents. There are different teams each week the with students. It is difficult to give any longitudinal feedback in that environment.” | Attending |
| Quotes on the feedback relationship as a form of coaching | |
| “I’ve thought long and hard about how we should be coaching. From day one it should be recognized as such… the coaching aspect of helping students meet their goals and articulate those goals is crucial.” | Attending |
| “If you have a good mentor, they should be helping you synthesize the information they’re getting from all different sources. It shouldn’t be just a subspecialty or research advisor but rather a clinical coach.” | Attending |
| Quotes on the importance of student-ownership over feedback | |
| “It’s on us to ask for feedback. That’s our job.” | Student |
| “[Feedback] works best when [the student] prompts you. It [must] be…driven by the person getting feedback. That burden needs to be on the student, and students [should be] empowered to do this.” | Resident |
| Quotes on how educators can set the stage for student-ownership over feedback | |
| “I’ve tried to put it on the learner a bit to identify one component they want to work on during their time with me. We can then use that as a specific point for feedback. This makes it a little more actionable, meaningful, and specific.” | Attending |
| “I like to have [the students] drive. I try to find as many opportunities for feedback as possible during normal conduct of teaching. This way, it’s in time that you do not need to devote to other responsibilities.” | Attending |
| “We have a very brief time with the students. It is important to focus on their experience as much as possible.” | Attending |
a Focus group quotations detailing the crucial relationship between Learner and Educator.
Students felt intimidated asking attendings for feedback, often perceiving residents as more accessible (Table 3). Some students felt residents are more entitled to attending-level feedback and considered this a reason resident feedback was reviewed more favorably. Faculty and learners suggested the ideal feedback model as considering the educator as a “coach”. They argued this model would better allow for continuity and follow-up. However, structural barriers like scheduling and rotation changes often disrupt these ideal dynamics (Table 3).
Focus groups frequently mentioned the importance of learner-initiated feedback (Table 3). There was agreement that feedback works best when students prompt for feedback, and even a sense that this is one of the learner’s responsibilities. Faculty pushed this a step forward in attempting to identify ways educators can set the stage for students to take ownership of feedback.
4.2.2. The Importance of Dedicated Time and Attention to Student Feedback
Focus groups emphasized the importance of allocating time for feedback (Table 4). Timely discussions, soon after observation, were seen as most effective. Participants suggested using brief windows in the clinical day or tracking observations to make feedback more specific and actionable.
Table 4.The Importance of Dedicated Attention to Student Feedback a
| Quotes | Focus Group Source |
|---|---|
| Quotes on time as a barrier to effective feedback | |
| “[Residents and attendings] are so busy that it is hard. They just do not have enough time.” | Student |
| “I think a lot of it boils down to just time. If you have time, you get more invested. If you’re more invested, you’re more willing to give better feedback.” | Student |
| “We don’t have time. The resident-as-teacher role is overtaken by the resident role of having to do all the tasks.” | Resident |
| “Faculty members must find a way to carve out time for the students to be observed. Faculty need protected time to give feedback.” | Attending |
| “Often very rushed – because everyone is super busy. This usually leads to it being very superficial.” | Attending |
| Quotes on keeping feedback timely and relevant | |
| “The way they do it now is through flash feedback that we go and collect from residents and attendings. It’s a way to give us more feedback in small chunks as we go. Though it can be…hard to get if the team is busy.” | Student |
| “It is a real challenge to remember the students – especially for summative evaluations. [When] you observe [them], you can give …concrete feedback. But it is… hard to remember these specifics even a week later.” | Attending |
| “The biggest issue is there’s a gap from when the feedback is given from when work was done. Catching attendings more quickly might be more specific or useful.” | Attending |
| “Some faculty write an index card for each student to help remember and jog the recollection [when feedback is delayed]. These are strategies which need to be worked on and developed.” | Attending |
a Focus group quotations regarding the need to put dedicated time and attention towards student feedback.
4.2.3. Ensuring Effective Feedback as an Expectation
During focus group sessions, participants were encouraged to provide suggestions on how to build an ideal feedback environment in a clinical setting. Participants emphasized that the best feedback occurs when there is clear communication that feedback is an inherent expectation and a part of the clinical workflow (Table 5).
Table 5.Ensuring Effective Feedback as an Expectation a
| Quotes | Focus Group Source |
|---|---|
| Quotes on feedback as an inherent expectation | |
| “Attending feedback that has been the most helpful is when they say proactively that we will do feedback at a given time. I find those attendings give the best feedback. If the attending knows the student will be with them for longer it tends to be more detailed.” | Student |
| “While students expect to receive feedback, attendings should expect to give it. It should be built into workflow.” | Student |
| “When I was a student attendings would come by in the afternoon every day for an hour. They would go over a topic and give us feedback. This way we had a steady relationship with them and knew it was coming.” | Resident |
| “I make it an expectation and have them show us components to give feedback on in real time. If more time could be allotted to rounds, there is more time to help the students learn.” | Attending |
| Quotes on ensuring educators and learners have clear and consistent definitions of feedback | |
| “There should be a least common denominator that carries through all the clerkships [making it] clear what the relationship to feedback is.” | Student |
| “I’m not sure if something standardized would be helpful... Maybe having some kind of standardized methodology, you would have students better understand how they are developing.” | Student |
| “[Educators are] always giving feedback. Even in the small moments of “did you check for x”. That’s feedback. You’re giving them some specific. thing that you are commenting on. They perceive that as teaching, but it’s really both. You just didn’t frame it as feedback. Unless you frame it that way, [students] will not recognize it as such… This is true for both residents and students… [Feedback is] a basic part of helping [them] learn to think and do neurology. Faculty are often giving a lot of feedback, even though [learners] may not realize that’s what they are doing.” | Attending |
| Quotes on the importance of formal training on feedback to ensure it embeds in the culture | |
| “We need to integrate how to teach and how to give feedback into residency education. Then it becomes more a part of the culture.” | Student |
| “We have more of an idea on how to give good feedback from [our own] bad experiences getting feedback.” | Resident |
| “I taught myself how to give feedback effectively because I am interested in doing that. I think it would be helpful if there was a format of how to give feedback. Something specific and objective, like a ‘smart phrase’.” | Resident |
| “Faculty development is necessary to help faculty involve the students more. It is just not a natural part of our workflow. We need to remember there is a student on the team who needs our attention as educators.” | Attending |
a Focus group quotations on building an expectation of feedback and embedding feedback in the culture of an educational environment.
Participants highlighted the need to clarify what constitutes feedback, as definitions often differ between learners and educators (Table 5). Standardized approaches across clerkships may help and allow expectations to permeate consistently across the curriculum. Additionally, both residents and faculty expressed a need for formal training to normalize feedback practices and embed them into clinical culture (Table 5).
4.2.4. Challenges Specific to Neurology Education
In discussing the neurology clerkship, faculty felt certain aspects of feedback were made more challenging by the nature of neurology as a specialty (Table 6). While the longer time spent with patients in a detailed evaluation can allow for building better relationships, it can also limit opportunities for supervisors to observe students’ abilities in totality. Residents pointed out the diverse spectrum of neurologic diagnoses and adjustments necessary to assess individual patients as a challenge to building cohesive and clear feedback. On the other hand, students had less appreciation of challenges specific to neurology and instead focused on attempts to streamline feedback across rotations to enhance their development (Table 6).
Table 6.Challenges Specific to Neurology Education a
| Quotes | Focus Group Source |
|---|---|
| “Neurology both makes it easier and harder. The intensive nature of a neurologic history and exam helps to allow students to establish relationships and spend time with patients, however the attending may not devote the necessary time to observing the full encounter and giving feedback.” | Attending |
| “One challenge on neurology is on consults when you’re so busy and seeing so many different patients. Each patient needs a different assessment. It can be hard to give consistent feedback when each case is so varied.” | Resident |
| “Overarching skills are quite similar between rotations. Maybe having some kind of standardized methodology, students would better understand how they are developing across the year.” | Student |
a Focus group quotations on perceived challenges specific to neurology education.
5. Discussion
Our results suggest that the format of feedback for students rotating in neurology may be less important than the nature of the relationship with the individual providing the feedback. Ratings of feedback before and after the intervention of a novel, self-reflective form were positive. However, a detailed review of students’ qualitative survey answers revealed significant issues in obtaining useful feedback. Students pointed out a lack of specificity, lack of time for feedback, and difficulty utilizing feedback to lead to actionable change. Interestingly, students rated feedback highly on a quantitative basis, despite qualitatively reporting numerous important barriers to effective feedback. There was notable concurrence between students, residents, and attendings in recognizing major issues with the current feedback system.
To improve medical student feedback in neurology, we recommend emphasizing strong educator-learner relationships with consistent expectations, prioritizing time for feedback, encouraging student-driven input, and training faculty in feedback delivery. This builds upon prior literature in further development of components and themes for feedback in neurology clerkships (21).
Students rated resident feedback as more effective than that from attendings, contrary to a traditional paradigm prioritizing seniority and experience. Focus groups suggested this was likely due to closer relationships between learner and educator. This challenges assumptions about seniority equating to better feedback and supports resident-led teaching in Neurology clerkships (22, 23).
Literature suggests the ideal dynamic regards a longitudinal coaching relationship (14, 24). Our study reinforces this with an emphasis on an educational alliance and dedicated time to foster mentorship. Attendings supported this concept; however, our findings suggest most current feedback models do not reflect it. Our novel feedback form aimed to promote student ownership but did not significantly enhance the feedback experience. This suggests format may be less important than relational dynamics. Effective clerkships should focus on building educator-learner connections, structured expectations, and consistent faculty training.
5.1. Conclusions
Our mixed-methods study assessed a self-reflective feedback tool and explored feedback dynamics in the neurology clerkship. Results highlight the importance of educator-learner relationships, dedicated attention to prioritize learner needs, and establishing consistent feedback expectations. Focusing on these objectives may address challenges to effective feedback, including lack of specificity, competing time commitments, and lack of training, which arise particularly in detail-oriented and time-consuming specialties such as neurology. Further educational interventions to optimize medical student feedback should place emphasis on these themes to promote specific, constructive, and actionable feedback. Additional work is needed to assess the applicability of these findings to other medical specialties.
5.2. Strengths
This study attempted to use mixed methods, including quantitative and qualitative analysis, to better understand the feedback dynamic in the neurology medical student clerkship. Furthermore, we solicited insight from a variety of stakeholders to better inform future efforts to improve medical student feedback in the neurology clerkship.
5.3. Limitations
There are several notable limitations, however. First, all data reviewed was self-reported. The content of feedback received pre- and post-implementation was not reviewed, and no objective measures were available to assess if feedback yielded a true change in behavior. Additionally, surveys were completed only by a small number of students with a limited response rate. We acknowledge that there may be different perceptions of feedback among students who did and did not complete the voluntary surveys and that the small sample size can limit our ability to detect statistical significance. Finally, we acknowledge that we have only assessed one approach to feedback, and the approach chosen may have biased responses to perceptions of feedback.
