https://doi.org/10.5812/aapm.106640

Relationship Between “Simulated Patient Scenarios and Role-Playing” Method and OSCE Performance in Senior Anesthesiology Residents: A Correlation Assessment Study

authors:

avatar Ali Dabbagh ORCID 1 , 2 , * , avatar Dariush Abtahi 1 , avatar Homayoun Aghamohammadi 1 , avatar Seyyedeh Narjes Ahmadizadeh ORCID 1 , avatar Seyed Hossein Ardehali 1 , avatar Anesthesiology Department 1

Anesthesiology Department, School of Medicine, Shahid Beheshti University of Medical Sciences, Tehran, Iran
Anesthesiology Research Center, Shahid Beheshti University of Medical Sciences, Tehran, Iran
Anesthesiology and Pain Medicine: Vol.10, issue 5; e106640
published online: August 22, 2020
article type: Research Article
received: June 23, 2020
revised: August 15, 2020
accepted: August 18, 2020

how to cite: Dabbagh A, Abtahi D, Aghamohammadi H , Ahmadizadeh S N, Ardehali S H, et al. Relationship Between “Simulated Patient Scenarios and Role-Playing” Method and OSCE Performance in Senior Anesthesiology Residents: A Correlation Assessment Study. Anesth Pain Med. 2020;10(5):e106640. https://doi.org/10.5812/aapm.106640.

Abstract

Background:

“Simulated-patient scenarios and role-playing” and OSCE are among the many non-traditional education methods with variable results in different clinical settings.

Objectives:

This cross-sectional study was performed to assess the correlation between the results of these two methods in senior anesthesiology residents, with a special focus on four of the six ACGME core competencies.

Methods:

During two years, senior anesthesiology residents were subject to “simulated patient scenario and role-playing” sessions. Two faculty members took the role of the patient and one of the relatives. An objective checklist with 15 items was prepared to be rated by other department faculty members. Meanwhile, an ordered pattern of OSCE was prepared to cover four core competencies that were more related to this academic process (from a total of six core competencies). The mean and standard deviation of the score of each of the 15 items in the checklist were calculated. The correlation between cumulative checklist scoring results and OSCE exam results was assessed. A P value of less than 0.05 was considered significant.

Results:

A total of 40 senior anesthesiology residents, with 344 assessments by faculty members in 40 sessions, were enrolled in the study. The questionnaire’s Cronbach’s alpha reliability was 0.74. No statistically significant disparity was detected between the results of the two assessment methods, while the results of the two assessments had a significant correlation (two-tailed correlation coefficient = 0.886; P value < 0.001).

Conclusions:

There was an objective relationship between the results of “simulated patient scenario and role-playing” strategies and the results of OSCE exams using an observer-based rating method. Thus, they could be used as surrogates in the assessment of core clinical competencies of senior anesthesiology residents.

1. Background

The core competencies defined by the Accreditation Council for Graduate Medical Education (ACGME) need an arsenal of medical education methods to thrive, especially in such stress loaded fields like anesthesiology (1-4). The “simulated-patient scenarios and role-playing” method is among the many non-traditional education methods with variable results in different clinical settings (5-7).

On the other hand, the Objective Structured Clinical Examination (OSCE) is a modern objective assessment technique in medical education with a systematic methodology and variable results. Although both “simulated-patient scenarios and role-playing” and OSCE methods are usually objective assessments, there is a paucity of data about any potential correlation in-between. In other words, there are not enough studies to demonstrate the correlation between the results of these two important methods (8-11). However, if we would find a significant correlation, this correlation between the results of the two assessment methods could provide evidence regarding the criterion validity (either in the form of concurrent validity if the two exams are rather simultaneous, or predictive validity if the gold standard is conducted later on).

In addition, some have expressed concerns regarding the validity and reliability of role-playing as an assessment method instead of a real situation (12-17). Considering the standards and international accord regarding the validity and reliability of OSCE, we sought to demonstrate if there is any objective relationship between the results of these two assessment methods. Thus, this study was designed and implemented to assess any potential relationship between these two objective assessment methods, considering that each of them would assess some ACGME core competencies (many in common). If such a hypothesis would be proven, the two methods can be used interchangeably after taking considerations regarding other aspects of validity, such as content validity.

2. Objectives

This observational analytic study was designed and performed to assess the correlation between the results of “simulated-patient scenarios and role-playing” and OSCE methods among senior clinical anesthesiology residents, with a special focus on four of the six ACGME core competencies (i.e., medical knowledge, practice-based learning and improvement, interpersonal and communication skills, and professionalism) in the Anesthesiology Department of Shahid Beheshti University of Medical Sciences (SBMU), Tehran, Iran. The four selected competencies were much more relevant in this study.

3. Methods

The study protocol was approved by the Institutional Review Board (IRB), Research Deputy, Shahid Beheshti University of Medical Sciences, Tehran, Iran. All the participants (i.e., residents who were assessed in the role-playing arm of the study) were informed that they were going to be assessed using this method. In addition, if any of them was reluctant to continue the study, his/her results were withdrawn from the study. Besides, none of the results was assessed in a personal manner. On the other hand, all of the faculty members who took part in the study filled out their scoring checklists anonymously, and they were free whether to take part in the study or not.

This cross-sectional study was designed and performed to evaluate the potential correlation between OSCE and role-playing results concerning the performance of anesthesiology residents. During two years (2018 and 2019 educational years), a general clinical case discussion class was weekly held each Monday afternoon. The senior anesthesiology residents (CA-4 or CA-3) were subject to “simulated patient scenario and role-playing” sessions. A real case scenario based on previous real patients was presented by a junior resident (either CA-1 or CA-2) that was supervised by an attending anesthesiologist who was in charge of the patient in the real clinical setting. The selection of the clinical topics was agreed upon the list of more common daily clinical challenges for the residents. The study sample included all CA-3 and CA-4 residents; indeed, there was no sampling, and all the residents were included in the study.

The role of the patient in the scenarios was played by two faculty members (a man and a woman), or if needed, one of the relatives of the patient, e.g., a parent or guardian of a child or a patient unable to talk directly. A checklist was prepared to assess the performance of senior anesthesiology residents (CA-4 or CA-3), which was standardized through the following steps (Table 1):

1. A primary draft was prepared based on previous studies, professionalism criteria, and interpersonal communication skills.

2. The face and content validity of the checklist was assessed by five faculty members of the Anesthesiology Department of SBMU.

3. A list with 15 items was finalized.

4. A five-point Likert scale was used for rating the 15 items, including “strongly agree = 5”, “agree = 4”, “neutral = 3”, “disagree = 2”, and “strongly disagree = 1”.

5. To test the questionnaire’s reliability, Cronbach’s alpha was calculated at the end of the study.

6. To improve the quality of the rating, the checklists were filled out anonymously. Although the faculty member knew the examinees and observed them during the test, they did not mention their names under the assessment checklist or did not sign it to compensate for one of the potential sources of bias.

Table 1.

Study Questionnaire Translated from Persian to English

The Stem of The QuestionStrongly AgreeAgreeNeutralDisagreeStrongly Disagree
Q1The resident adhered strictly to ethical principles while talking to the patient (or patient companion).
Q2The resident showed enough self-esteem in dealing with the patient (or patient companion)
Q3The resident spent enough time to take the history of the disease.
Q4The resident performed necessary examinations during the patient visit.
Q5The resident delayed the patient more than usual before performing anesthesia on the operating room bed.
Q6The resident induced unnecessary excessive stress to the patient (or patient companion)
Q7The resident correctly presented his/her professional and scientific status to the patient (or patient companion).
Q8The resident communicated the information calmly and realistically when declaring the patient's clinical condition.
Q9The resident conveyed to the patient (or patient companion) sufficient information about anesthesia.
Q10The resident provided the patient (or patient companion) with information about the procedure.
Q11Medical explanations provided by the resident to the patient (or patient companion) were unnecessary.
Q12The resident provided the patient (or patient companion) with adequate explanations of possible complications and risks in the perioperative period.
Q13The resident recorded informed consent in the patient's file.
Q14The resident coordinated with the responsible anesthesiologist for anesthesia.
Q15The resident introduced the anesthesia team (including responsible attending, other anesthesia residents, the anesthesia nurse) to the patient (or patient companion).

The performance of each senior anesthesiology resident (CA-4 or CA-3) was rated as follows:

1. The simulated patient scenario was presented by a CA-1 or CA-2 resident supervised by a faculty member.

2. One of the faculty members played the role of the simulated patient who was a candidate for an elective or emergent anesthesia plan to undergo a surgical procedure.

3. Another faculty member played the role of “a patient’s relative” or “a guardian of a child” in the sessions that were to assess pediatric patients.

4. The senior anesthesiology resident (CA-4 or CA-3) interviewed and handled the patient.

5. Those faculty members who observed the session rated the senior anesthesiology resident (CA-4 or CA-3) objectively and anonymously, based on the finalized checklist (Table 1).

6. At least seven faculty members of the Anesthesiology Department of SBMU took part in each of these rating sessions. There was no upper limit for the observer faculty members to rate the checklist, as, in one of the sessions, a ceiling of 19 faculty members was touched.

Meanwhile, an ordered pattern of OSCE was defined by the technical team of the Education Deputy, Anesthesiology Department, SBMU, to cover the four competencies, i.e., medical knowledge, system-based practice, interpersonal and communication skills, and professionalism. This OSCE was prepared in 10 stations, focusing on a range of clinical common challenges, covering the daily challenges of anesthesiology residents. The OSCE was performed exactly in the midterm of the study, i.e., April 2019.

The study outcomes and variables are as follows:

● Study outcomes: The relationship between OSCE scores and the scores of faculties on the role-playing checklist.

● Exposure factors: In this study, several exposure factors were defined including the exposure variable that was the time duration for the study, which could affect the outcome, since, with time, the knowledge of the residents may have been affected; this might have effects on the outcome (assessment results). Besides, the personal attitudes of the faculty members (i.e., their attitudes) towards each trainee could be the exposure factor. However, since there was a global assessment of “role-playing” scores and they were then compared with the OSCE results, the effects of the above factors were possibly minimized, resulting in the alleviation of both non-differential and differential measurement errors (18).

● Predictors: The general performance of each resident in previous formal assessments could be considered a predictor for the results of OSCE and role-playing (19).

● Potential confounders and effect modifiers: They included the atmosphere of the role-playing session.

The mean and standard deviation of each of the 15 items in the checklist were calculated. All checklist scoring results were also accumulated to calculate the correlation between the checklist assessments and the semester-length scores. Another correlation was calculated between the scores on the “simulated patient scenario and role-playing” checklists gained by each resident and the individual OSCE exam results; in other words, the results of each resident on “simulated patient scenario and role-playing” were compared with his/her results on the departmental OSCE exam. To compare the results, both the results of the OSCE exam and the scores of the “simulated patient scenario and role-playing” checklist were calculated based on “hundred scores”. Usually, the assessment ranks are performed with different scoring scales; however, we aimed to compare the results of the two different assessment methods (i.e., OSCE scores and the scores of “simulated patient scenario and role-playing” checklist) with each other using different scales. Therefore, using a mathematical calculation, we adjusted the results of the two assessment methods to make the comparisons easier.

4. Results

From 2018 to 2019, a total of 40 senior anesthesiology residents (CA-4 or CA-3) entered the study (CA-4 = 19 residents, CA-3 = 21 residents). Besides, 344 assessments done by faculty members were collected in 40 sessions, with a minimum of 7 and a maximum of 19 faculty members who took part in rating the senior residents, yielding a mean of 8.6 raters per each senior resident (i.e., 8.6 faculty members rated in each session). The questionnaire’s Cronbach’s alpha reliability was 0.74.

There was no statistically significant difference between the results of the aggregated OSCE exam (71 ± 17) and the scores of the “simulated patient scenario and role-playing” checklist (74 ± 16) using the independent t-test as cumulative results (P value < 0.01). Of course, as mentioned in the methods section, both the results of the OSCE exam and the scores of the “simulated patient scenario and role-playing” checklist were modified and calculated on the basis of “hundred scores” to facilitate the comparisons.

In addition, for each of the senior anesthesiology residents (CA-4 or CA-3), there was an individual significant correlation between the results of the OSCE exam and the scores of the “simulated patient scenario and role-playing” checklist. In addition, there was a similar correlation for the cumulative results of the two methods (two-tailed correlation coefficient = 0.886; N = 40; P value < 0.001).

5. Discussion

This study demonstrated that the “simulated patient scenario and role-playing” method in a departmental educational session, when done in a regular pattern, is in strong concordance with OSCE results of the same population. Considering this finding and keeping the role of OSCE in mind, we may hypothesize that simulated patient scenario and role-playing sessions could improve the performance of the anesthesiology residents regarding the ACGME core competencies, a hypothesis mandating a different methodology and another study (such as a pre- and post-design study). However, for gaining such a degree of generalizability (external validity) of the study results, more complementary studies are mandatory.

In addition, there was a good correlation between the OSCE examination scores and the scores of simulated patient scenarios and role-playing sessions. This is while both of these assessments were done using objective measures. The questionnaire reliability (Cronbach’s alpha) was at an appropriate level, indicating an acceptable reliability level of the rating checklist.

The six core competencies of ACGME include patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, and systems-based practice. Among the six core competencies, the four latter competencies were much more relevant in the study, and this is why we chose them (2, 3, 20). Throughout the clinical practice, these competencies that focus far beyond just medical knowledge should be built up using a systematic training approach. In addition, they should be measured in a formative assessment and robust evaluation model. The latter approach would lead to an appropriate process of “clinical thriving” in medical education of residency programs. These studies are in favor of our research; however, some studies mandate further assessments to evaluate the psychomotor and behavioral skills of trainees (7, 21).

However, in some anesthesiology residency programs, a try and error pattern is used instead of a systematic approach, which would not thrive the capacities needed for trainees, including professionalism, especially considering the relatively high stress loaded fields of clinical anesthesiology (1-3, 7, 22). This is why competency-based medical education is much more appreciated with its special focus on outcomes, capabilities, and learner-centeredness, emphasizing the final outcome of medical education. This is a specific feature that was assessed in our study and has been the focus of some previous researches, being in favor of our findings (23-27).

Although several simulation methods have been developed, the “simulated patient scenario and role-playing” method has been described as a cost-effective method with fruitful results in training. This notion supports our findings in this study, although some aspects of our study need more attention (5, 6, 28-30).

Teaching medical emergencies using a simulation-based model has been demonstrated as an effective method in improving students’ knowledge, experience, and confidence, for the management of critical events (7, 31, 32). Role-playing is one of those techniques benefiting from simulation-based models, leading to improvement in several aspects of ACGME core competencies, especially those affected by a load of stress in anesthesiology (7, 32-34). As pointed out by Dias et al., “Role-play strategies can provide high psychological fidelity for simulation-based training” (35). Besides, there are practical clinical outcomes in patient care, leading to tangible results (28, 29), although some controversies exist (30). In addition, Lambert et al. demonstrated that the Emotional Quotient (EI)-related capabilities of anesthesiology residents through simulation-based medical education would improve important aspects of ACGME competencies, promoting patient outcomes in perioperative care (7).

5.1. Study Limitations

Among the study limitations, the following could be mentioned as the main limitations:

1. Simulated patient scenarios and role-playing are techniques that can be used in teaching and learning activities or assessments. They are not stand-alone methods by themselves. In addition, role-playing and patient scenarios could be moved to an OSCE station. However, in this study, we compared the two methods that are inherently different; indeed, what we performed in this study was the comparison between the results of an assessment performed through single observation of one examinee by several examiners who scored based on a pre-designated checklist and the results of an OSCE. The former is very close to a “long case” exam, but on a simulated patient, it is not as a workplace-based assessment. Considering the settings of our study, there are some similarities between our research and an objective structured clinical viva examination (OSVE).

2. The so-called “scenario-based simulation and role-playing” method was conducted over a span of about two years, while each resident did not take part in the process repeatedly. The knowledge and skills of the examinees were bound to change over this rather long period. Each resident was evaluated only once (using the simulation model). Thus, it could have been very early or very late in the study period, and they could not be assessed in a more sophisticated way; however, the goal of the research was to perform a global and collective movement involving all residents, which may partly compensate for this gap.

3. Only four of the six ACGME core competencies were assessed; however, this was due to the design of the study. Future studies should focus on other aspects of ACGME core competencies.

4. Role-playing was the only strategy used in this research; however, other strategies and modalities used for experiential learning were not considered in this study, which should be focused on in future studies to cover the gap.

In conclusion, though there are many studies considering the simulated patient scenario and role-playing approach, scant studies have assessed the relationship between “simulated patient scenario and role-playing” strategies and the results of OSCE exams using an observer-based rating method. Our study could demonstrate an objective relationship between the results of these two assessment methods. Thus, based on our findings, these two methods could be used as surrogates for each other in the assessment of the core clinical competencies of senior anesthesiology residents.

Contributor Information

Noor Mohammad Arefian, Sogol Asgari ORCID , Hamidreza Azizi-Farsani, Faranak Behnaz ORCID , Shideh Dabir ORCID , Payman Dadkhah, Mastaneh Dahi ORCID , Masih Ebrahimy Dehkordy ORCID , Ahmad Eghbali, Hedayatollah Elyasi, Lida Fadaizadeh, Kamal Fani ORCID , Mohammad Fathi, Behrooz Farzanegan, Mehdi Ghahremani, Babak Gharaei ORCID , Mahshid Ghasemi ORCID , Reza Goharani ORCID , Masoud Hashemi, Seyed Mohammad Reza Hashemian, Alireza Jaffari ORCID , Hossein Ali Jelveh-Moghadam, Mohammadreza Hajiesmaeili ORCID , Behnam Hoseini ORCID , Morteza Jabbary Moghaddam, Alireza Jahangiri Fard, Mohammadreza Kamranmanesh ORCID , Shayesteh Khorasanizadeh ORCID , Mehran Kouchek ORCID , Homeyra Kouzekanani, Nilofar Massoudi ORCID , Alireza Mahdavi, Nastaran Sadat Mahdavi, Elham Memary, Alireza Mirkheshti, Mir-Mohammad Miri, Gholamreza Mohseni, Sirous Momenzadeh ORCID , Faramarz Mosaffa ORCID , Mohammadreza Moshari ORCID , Kamran Mottaghi, Arash Najafi Abrandabadi, Masoud Nashibi ORCID , Navid Nooraei, Hossein Nouri, Davood Ommi, Tahereh Parsa, Mahtab Poorzamani, Badiozaman Radpay, Seyed Sajad Razavi, Fatemeh Roodneshin, A. Sassan Sabouri ORCID , Afsaneh Sadeghi, Farhad Safari ORCID , Sara Salarian ORCID , Alireza Salimi, Sohrab Salimi ORCID , Shahram Sayyadi ORCID , Parissa Sezari ORCID , Seyedpouzhia Shojaei ORCID , Sedigheh Shokrollahi, Soodeh Tabashi ORCID , Mehrdad Taheri ORCID , Ardeshir Tajbakhsh ORCID , Houman Teymourian, Maryam Vosoghian ORCID

References

  • 1.

    Chestnut DH. On the Road to Professionalism. Anesthesiology. 2017;126(5):780-6. [PubMed ID: 28187021]. https://doi.org/10.1097/aln.0000000000001545.

  • 2.

    Colbert-Getz JM, Lappe K, Northrup M, Roussel D. To What Degree Are the 13 Entrustable Professional Activities Already Incorporated Into Physicians' Performance Schemas for Medical Students? Teach Learn Med. 2019;31(4):361-9. [PubMed ID: 30873878]. https://doi.org/10.1080/10401334.2019.1573146.

  • 3.

    Fleischut PM, Evans AS, Nugent WC, Faggiani SL, Kerr GE, Lazar EJ. Perspective: call to action: it is time for academic institutions to appoint a resident quality and patient safety officer. Acad Med. 2011;86(7):826-8. [PubMed ID: 21617508]. https://doi.org/10.1097/ACM.0b013e31821da286.

  • 4.

    Dabbagh A, Sezari P, Tabashi S, Tajbakhsh A, Massoudi N, Vosoughian M, et al. Attitudes of anesthesiology residents towards a small group blended learning class. Anesth Pain Med. 2020;10(3). e103148.

  • 5.

    Ichikawa N, Narita M, Yamada Y. Role Playing of Predischarge Family Conference as an Educational Approach of Regional Palliative Care for Medical Profession: A Pilot Study. J Palliat Med. 2016;19(10):1031. [PubMed ID: 27574871]. https://doi.org/10.1089/jpm.2016.0248.

  • 6.

    Johansson J, Skeff KM, Stratos GA. A randomised controlled study of role play in a faculty development programme. Med Teach. 2012;34(2):e123-8. [PubMed ID: 22289010]. https://doi.org/10.3109/0142159x.2012.644832.

  • 7.

    Lambert JA, Vanderbilt AA, Papadimos TJ. Improved emotional intelligence in perioperative care through simulation-based medical education during anesthesiology residency training: a call for implementation. Adv Med Educ Pract. 2019;10:39-42. [PubMed ID: 30774498]. [PubMed Central ID: PMCPmc6361314]. https://doi.org/10.2147/amep.s164752.

  • 8.

    Warner DO, Isaak RS, Peterson-Layne C, Lien CA, Sun H, Menzies AO, et al. Development of an Objective Structured Clinical Examination as a Component of Assessment for Initial Board Certification in Anesthesiology. Anesth Analg. 2020;130(1):258-64. [PubMed ID: 31688077]. https://doi.org/10.1213/ane.0000000000004496.

  • 9.

    Rebel A, Dilorenzo A, Isaak R, McGrane S, Moran KR, Mobley EC, et al. Replicating an Educational OSCE Project for Skill Assessment of Junior Anesthesiology Residents at Multiple Institutions: A Qualitative Description. J Educ Perioper Med. 2018;20(2). E622. [PubMed ID: 30057933]. [PubMed Central ID: PMCPmc6055536].

  • 10.

    Robertson AC, Fowler LC, Niconchuk J, Kreger M, Rickerson E, Sadovnikoff N, et al. Application of Kern's 6-Step Approach in the Development of a Novel Anesthesiology Curriculum for Perioperative Code Status and Goals of Care Discussions. J Educ Perioper Med. 2019;21(1). E634. [PubMed ID: 31406705]. [PubMed Central ID: PMCPmc6685461].

  • 11.

    Miller C, Toy S, Schwengel D, Isaac G, Schiavi A. Development of a Simulated Objective Structured Clinical Exam for the APPLIED Certification Exam in Anesthesiology: A Two-Year Experience Informed by Feedback from Exam Candidates. J Educ Perioper Med. 2019;21(4). E633. [PubMed ID: 32123698]. [PubMed Central ID: PMCPmc7039675].

  • 12.

    Cohen DS, Colliver JA, Marcy MS, Fried ED, Swartz MH. Psychometric properties of a standardized-patient checklist and rating-scale form used to assess interpersonal and communication skills. Acad Med. 1996;71(1 Suppl):S87-9. [PubMed ID: 8546794]. https://doi.org/10.1097/00001888-199601000-00052.

  • 13.

    Cleland JA, Abe K, Rethans JJ. The use of simulated patients in medical education: AMEE Guide No 42. Med Teach. 2009;31(6):477-86. [PubMed ID: 19811162]. https://doi.org/10.1080/01421590903002821.

  • 14.

    Jha V, Setna Z, Al-Hity A, Quinton ND, Roberts TE. Patient involvement in teaching and assessing intimate examination skills: a systematic review. Med Educ. 2010;44(4):347-57. [PubMed ID: 20444070]. https://doi.org/10.1111/j.1365-2923.2009.03608.x.

  • 15.

    Nestel D, Tierney T. Role-play for medical students learning about communication: guidelines for maximising benefits. BMC Med Educ. 2007;7:3. [PubMed ID: 17335561]. [PubMed Central ID: PMC1828731]. https://doi.org/10.1186/1472-6920-7-3.

  • 16.

    Swartz MH, Colliver JA, Cohen DS, Barrows HS. The effect of deliberate, excessive violations of test security on performance on a standardized-patient examination. Acad Med. 1993;68(10 Suppl):S76-8. [PubMed ID: 8216641]. https://doi.org/10.1097/00001888-199310000-00052.

  • 17.

    Barrows HS. An overview of the uses of standardized patients for teaching and evaluating clinical skills. AAMC. Acad Med. 1993;68(6):443-51. discussion 451-3. [PubMed ID: 8507309]. https://doi.org/10.1097/00001888-199306000-00002.

  • 18.

    Lee TA, Pickard AS; et al. Exposure Definition and Measurement. In: Velentgas P, Dreyer NA, Nourjah P, editors. Developing a Protocol for Observational Comparative Effectiveness Research: A User's Guide. Chapter 4. Rockville (MD): Agency for Healthcare Research and Quality (US); 2013.

  • 19.

    Bierman K. Predictor variables associated with positive Fast Track outcomes at the end of third grade. J Abnorm Child Psychol. 2002;30(1):37-52. [PubMed ID: 11930970]. [PubMed Central ID: PMC2756765].

  • 20.

    Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990;65(9 Suppl):S63-7. [PubMed ID: 2400509]. https://doi.org/10.1097/00001888-199009000-00045.

  • 21.

    Natesan P, Batley NJ, Bakhti R, El-Doueihi PZ. Challenges in measuring ACGME competencies: considerations for milestones. Int J Emerg Med. 2018;11(1):39. [PubMed ID: 31179910]. [PubMed Central ID: PMCPmc6326150]. https://doi.org/10.1186/s12245-018-0198-3.

  • 22.

    Dabbagh A, Massoudi N, Vosoghian M, Mottaghi K, Mirkheshti A, Tajbakhsh A, et al. Improving the Training Process of Anesthesiology Residents Through the Mentorship-Based Approach. Anesth Pain Med. 2019;9(1). e88657. https://doi.org/10.5812/aapm.88657.

  • 23.

    Bok HG, Teunissen PW, Favier RP, Rietbroek NJ, Theyse LF, Brommer H, et al. Programmatic assessment of competency-based workplace learning: when theory meets practice. BMC Med Educ. 2013;13:123. [PubMed ID: 24020944]. [PubMed Central ID: PMCPmc3851012]. https://doi.org/10.1186/1472-6920-13-123.

  • 24.

    Williams BW, Byrne PD, Welindt D, Williams MV. Miller's Pyramid and Core Competency Assessment: A Study in Relationship Construct Validity. J Contin Educ Health Prof. 2016;36(4):295-9. [PubMed ID: 28350312]. https://doi.org/10.1097/ceh.0000000000000117.

  • 25.

    Ten Cate O. Competency-Based Postgraduate Medical Education: Past, Present and Future. GMS J Med Educ. 2017;34(5):Doc69. [PubMed ID: 29226237]. [PubMed Central ID: PMCPmc5704607]. https://doi.org/10.3205/zma001146.

  • 26.

    Bisgaard CH, Rubak SLM, Rodt SA, Petersen JAK, Musaeus P. The effects of graduate competency-based education and mastery learning on patient care and return on investment: a narrative review of basic anesthetic procedures. BMC Med Educ. 2018;18(1):154. [PubMed ID: 29954376]. [PubMed Central ID: PMCPmc6025802]. https://doi.org/10.1186/s12909-018-1262-7.

  • 27.

    Weller JM, Naik VN, San Diego RJ. Systematic review and narrative synthesis of competency-based medical education in anaesthesia. Br J Anaesth. 2020. [PubMed ID: 32008702]. https://doi.org/10.1016/j.bja.2019.10.025.

  • 28.

    Bosse HM, Nickel M, Huwendiek S, Schultz JH, Nikendei C. Cost-effectiveness of peer role play and standardized patients in undergraduate communication training. BMC Med Educ. 2015;15:183. [PubMed ID: 26498479]. [PubMed Central ID: PMCPmc4619415]. https://doi.org/10.1186/s12909-015-0468-1.

  • 29.

    Dabbagh A, Elyassi H, Sabouri AS, Vahidshahi K, Ziaee SAM; Anesthesiology Department. The Role of Integrative Educational Intervention Package (Monthly ITE, Mentoring, Mocked OSCE) in Improving Successfulness for Anesthesiology Residents in the National Board Exam. Anesth Pain Med. 2020;10(2). e98566. [PubMed ID: 32547933]. [PubMed Central ID: PMCPmc7260396]. https://doi.org/10.5812/aapm.98566.

  • 30.

    Bray BS, Schwartz CR, Odegard PS, Hammer DP, Seybert AL. Assessment of human patient simulation-based learning. Am J Pharm Educ. 2011;75(10):208. [PubMed ID: 22345727]. [PubMed Central ID: PMCPmc3279041]. https://doi.org/10.5688/ajpe7510208.

  • 31.

    Marti K, Sandhu G, Aljadeff L, Greene R, Lesch AB, Le JM, et al. Simulation-Based Medical Emergencies Education for Dental Students: A Three-Year Evaluation. J Dent Educ. 2019;83(8):973-80. [PubMed ID: 30962311]. https://doi.org/10.21815/jde.019.084.

  • 32.

    Weinger MB, Banerjee A, Burden AR, McIvor WR, Boulet J, Cooper JB, et al. Simulation-based Assessment of the Management of Critical Events by Board-certified Anesthesiologists. Anesthesiology. 2017;127(3):475-89. [PubMed ID: 28671903]. https://doi.org/10.1097/aln.0000000000001739.

  • 33.

    Karam VY, Barakat H, Aouad M, Harris I, Park YS, Youssef N, et al. Effect of a simulation-based workshop on breaking bad news for anesthesiology residents: an intervention study. BMC Anesthesiol. 2017;17(1):77. [PubMed ID: 28615002]. [PubMed Central ID: PMCPmc5471713]. https://doi.org/10.1186/s12871-017-0374-7.

  • 34.

    Bhagwat M. Simulation and anaesthesia. Indian J Anaesth. 2012;56(1):14-20. [PubMed ID: 22529414]. [PubMed Central ID: PMCPmc3327064]. https://doi.org/10.4103/0019-5049.93338.

  • 35.

    Dias RD, Scalabrini-Neto A. Acute stress in residents playing different roles during emergency simulations: a preliminary study. Int J Med Educ. 2017;8:239-43. [PubMed ID: 28658657]. [PubMed Central ID: PMCPmc5511741]. https://doi.org/10.5116/ijme.5929.60f1.