1. Background
Mentorship plays a crucial role in professional education and academic development; which has been used in a number of residency programs worldwide (1); however, the final accepted-method for “anesthesiology resident mentorship” still needs more research (2). Although many accredited anesthesiology programs benefit from mentorship as a quality improvement tool, the most efficient methods for maximizing the educational goals of mentorship remains to be defined yet. Thus a number of challenging issues seem unanswered before reaching the clear ultimate goals of mentorship (3, 4).
2. Objectives
This study investigates the role of an “evaluation and feedback” mentorship program for anesthesia residents; which has been completed through a combination of sequential methods in Anesthesiology Department, Shahid Beheshti University of Medical Sciences, including:
1- Reporting and feedback
2- Evaluation through multiple stepwise formative assessments
3- Role-playing for stress management
4- Creating a “supportive atmosphere” for the residents
3. Methods
The main objective of the mentoring program was to evaluate the outcome of improving the scientific preparation of anesthesiology residents with the ultimate goal of creating a supportive atmosphere to improve professional counseling. New mentoring programs highlighted student-faculty association and professional career counseling; while emphasizing health, reducing stress, and planning student activities. Mentoring programs focused on two main goals:
1. Our first main goal in implementing the mentoring program was “the need to provide residents with a provision of a mentor”; which would provide residents with a supportive environment to improve their theoretical knowledge, clinical skills, and professional capacities, through role simulations and improvement in professional management.
2. The other aims of our mentoring programs were constructing a “safe environment” and a “supportive and creative atmosphere” that would encourage and foster reflection, promotion, self-care, and sense of wellness, monitoring personal improvement, afford resource for residents pursuing control, improve construction and problem-solving team, and assisting in career counseling.
All the nine educational wards of Anesthesiology Department, Shahid Beheshti University of Medical Sciences took part in the “resident mentorship program” from October 2016 to December 2018. Nine faculty members participated voluntarily as core members of the program. The program was a part of a completely supportive and educational package with the following items:
1. Creating online groups for communicating, feedbacks, idea sharing and problem-solving
2. Preparing a didactic list of chapters based on the textbooks announced as references for the national board exam
3. Integrating each of the chapters based on a scheduled list of the program according to a designed 6-months schedule
4. Designing a comprehensive plan of assessments throughout the semester, including both formative and summative assessments; i.e., patient management problem (PMP), objective structured clinical examination (OSCE) and scheduled multiple-choice exams based on the didactic program of chapters
5. Programmed and scheduled role-playing as a task for improving communication skills and to enhance stress management skills for CA-4 anesthesiology residents
6. Creating a volunteer group from attending professors entitled the “Anesthesiology Department Examination Committee: ADEC”
At the end of each month, an official multiple choice question exam was taken with one hundred and fifty questions; the ADEC was responsible for holding the exam from A to Z; i.e.
• Designing the exam calendar
• Scheduling the didactic content of the exam
• Preparing the exam questions considering content validity and face validity of each item test
• Analyzing the exam results both per question and per examinee (i.e. each resident)
• Giving feedbacks regarding test results to each resident, comparing each individual exam with other exam series for each of the residents and also, each resident with other peers within each exam
• Giving feedback to each anesthesiology department faculty member regarding the overall score of the designed tests (in terms of difficulty index and discrimination score); each faculty member designed and prepared 5-10 tests for each monthly exam
• Meanwhile, a constant member of the ADEC was committed to preparing the tests as a booklet; with scientific discussions and explanations about correct and incorrect answers based on the didactic textbooks; this booklet was provided to each resident when the exam was finished in order to perform self-assessment.
• At the end of each monthly exam, each of the residents was free to submit his/her viewpoints regarding the whole program, including his/her mentorship process and/or other educational issues in an anonymous manner; these feedbacks were reviewed directly by the chairman of the anesthesiology department.
Each resident has to pass a definite level in order to pass to the next clinical residency year in the annual promotion exam of the IBAITE (Iranian Board of Anesthesiology In-Training Examination).
The three educational year periods were compared regarding the annual IBAITE exam scores for each of CA-1 to CA-4 level residents and the final performance of the IBA (Iranian Board of Anesthesiology) in 3 consecutive years:
• 2014 to 2015
• 2016 to 2017
• 2018 to 2019
The detailed comparison of IBAITE scores throughout these years was shown in Table 1.
Table 1.
Comparison of Iranian Board of Anesthesiology In-Training Examination (IBAITE) Scores by Clinical Anesthesia Year (CA)a
| Period 1b | Period 2c | P Valued | |
|---|---|---|---|
| CA-1 | 50.63 ± 4.8 | 57.70 ± 4.9 | < 0.001 |
| CA-2 | 57.10 ± 4.5 | 62.04 ± 4.9 | 0.002 |
| CA-3 | 59.44 ± 6.3 | 65.10 ± 5.3 | 0.009 |
| CA-4 | 69.70 ± 5.4 | 80.00 ± 5.1 | 0.002 |
aValue are expressed as mean ± SD scores, scores range:0 - 90.
bPeriod 1: 2015 - 2016.
cPeriod 1: 2016 - 2017.
dP value < 0.05 considered statically significant.
Study data were analyzed using SPSS software (version 18, IBM Corp, Chicago, IL, USA). Average IBAITE scores were calculated for each resident and the periodic differences were determined using the IBAITE scores and the IBA written QE passing rate. Linear regression analysis was used to assess the effect of exposure to the educational program on the educational achievement indices, in which exposure time interval was defined as the years of the intervention program.
Mean values of continuous variables were calculated. Changes in IBAITE mean scores between three periods were analyzed using t-test. To compare the difference in written QE, Chi-square odds ratios were calculated for passing rates between the 3 periods. Any P values less than 0.05 was considered statistically significant.
4. Results
Sixty-seven residents were evaluated and IBAITE was taken over the two consecutive years of the study periods. Histograms show the distribution of residents scores in the IBAITE during 2 periods of the study in Figure 1.
Figure 1.
Histograms of the distribution of resident IBAITE and evaluation scores: A, mean IBAITE scores in period 1; B, mean IBAITE scores in period 2
The mean ± standard deviation for IBAITE total score in all clinical anesthesia (CA) levels in period 2 was 63.58 ± 8.3 and it was significantly higher than period 1 (57.11 ± 7.8) (P < 0.05).
There were no differences in the mean IBAITE total score between 2014 and 2015 but there was a significant increase in IBAITE total score in 2017 in comparison to 2016 (P < 0.05).
The largest increase in mean scores occurred among the residents of CA-4, with an average percentage score of 69.70 ± 5.4 to 80.00 ± 5.1. The slightest increase in mean scores was distinguished among the CA-2 residents, whose mean score was augmented only from 57.10 ± 4.5 to 62.04 ± 4.9.
In pre-board exam (IBAITE for CA-4 residents) there was a significant increase of passing rate in period 2 in comparison to period 1 (95% vs 50%, P < 0.05). IBAITE scores were compared throughout these years and were shown in Table 1.
In the first period (2014 - 2015), 14 residents graduated from our department. During this period, just 7 (50%) residents passed the IBA written QE in the first attempt. In the second period (2015 - 2017), after instituting the educational program, 12 of 14 residents [85%, odds ratio (OR) 1.7 (95% CI 0.97 - 3.01) versus course 1] passed IBA QE written at first try. The residents in period 2 were exposed to the educational intervention for at least 1 year. Also, there was a significant correlation between the change in mean IBAITE scores in period 1 and IBA written QE passing rate in the first attempt (r = 0.55, P < 0.05).
Furthermore, a linear regression model showed that the IBAITE score was improved by 3.7 points per year exposure to the educational program (P < 0.05).
5. Discussion
This study demonstrated that applying the mentorship program could result in improved performance of anesthesiology residents regarding both professional skills and theoretical assessments. This study was designed and performed in order to assess the role of mentorship in improving clinical skills and theoretical knowledge; emphasizing on mentorship as an integral part of role-modeling technique; especially when considering the paucity of definitive results (3, 5).
From the viewpoint of trainees, the main objectives in all residency programs are to learn clinically important skills, improve the theoretical needed knowledge, and to learn the professional behaviors by practice. Mentorship has been shown to improve all these main goals in a number of studies; though not all of them. Our study, however, demonstrated an integral and significant role for mentorship in achieving these goals (2-4).
On the other hand, the low rates of participation of faculties in mentorship activity are one of the main problems in implementing mentoring, which might question its effectiveness (3). There is a growing demand for further sophisticated research on the main goals and the professional needs of anesthesiology residents. This is of special concern when considering the mentorship program as a more practical assessment method for current residency programs; in such a way that the best appropriate structure of a mentoring program could be determined. Our study, compared with other similar ones, provided an objective and verified tool tailored for anesthesiology residents (2). When creating a mentor-mentee relationship, it is mandatory for both sides (i.e. mentor and mentee) to make consensus on main goals in order to create a long term professional program (6). In this setting, the role of program administrator is an essential one and is considered to be the cornerstone of the process; mainly because his/her administrative role guarantees the success of the mentorship program. Also, in many instances, the program director besides being involved in leadership, often provides a coordinative role in order to prevent any potential challenge between characters of mentees and mentors (4).
In terms of outcomes, there was an incremental increase in resident achievement rates in each academic year regarding the pass level of the National Board Exam annual rating scales with a significant annual growth rate. This finding was in favor of the effects of the mentorship program, especially when comparing with some similar studies, which demonstrated the mentorship program as an opportunity for training achievements and also, as a method to improve learning new clinical skills and professionalism in the residents (7).
During the last years, the very rapid improvements in medicine have augmented the extent and importance of “must learn” topics for trainees in all fields of medicine, necessitating novel approaches for coping with the educational challenges. In the mentorship model, not only theoretical topics are concerned but also clinical skills and professional capacities are considered the main goals of education, which are often ignored in traditional methods of residency education (8-11); among them, the following capacities and skills could be mentioned:
• Clinical discussions
• Stress management
• Capacity to cope with failure in educational level achievement
• Inability to plan a long term study design
• Integrating content of textbooks
• Misconduct protection in academic achievements
• Stepping down plans
In conclusion, our study demonstrated the constructive role of a mentorship program in education anesthesiology residents, which was well designed, well assessed, and presented a teamwork; leading to significant improvements in educational goals. The results of this study demonstrated clinical mentorship program as an effective method in improving theoretical, clinical, and professional achievement of anesthesiology residents.
