1. Background
Medical education is constantly changing. In the Edinburgh Declaration (1988), the World Federation of Medical Education (WFME) announced (1): “The aim of medical education is to produce doctors who will promote the health of all people, and that aim is not being realized in many places, despite the enormous progress in the bio-medical sciences. These defects have been identified for a long time, but efforts to introduce greater social awareness in medical schools have not been notably successful.” At the beginning of the new millennium, Rees (2) still believes that medical education has this challenge that” how should doctors be trained to perform their tasks properly?”
The Shahid Beheshti Medical School (SBMS) also recognized this need and, following a partnership with the WFME in developing the undergraduate medical education (UME) global standards, introduced the big reform in its general practitioner (GP) curriculum that was honored by the WFME visiting board in Iran in 2003 (3, 4). After more than 11 years of implementing the reformed program and conducting multiple evaluations, it was time for a graduate survey.
It is logical that an educational program is evaluated, at least in part, by the performance of its graduates to provide evidence for the effectiveness of the program (5). Graduates’ feedback to the educational program as a reasonable approach to evaluation is used in different parts of the world, such as Canada, US (6), Taiwan (7), and Iran (6). The Association of American Medical Colleges (AAMC) has used its graduation questionnaire for surveying medical students in the US since 1978 as a source of feedback to the programs and showed that student perceptions have greater longitudinal stability (7). Some believe that graduates have a unique understanding of program quality, and program directors can use their feedback as a valuable source to improve their programs (6).
The Shahid Beheshti General Medical Degree curriculum-2004 (SBGMD curriculum-2004), which has been running since 2004, is a medical doctor (MD) program of SBMS whose graduates can practice as an independent GP in Iran. This program is a seven-year curriculum with four phases: Basic sciences phase (2.5 years), introduction to clinical sciences/preclinical phase (six months), clinical sciences phase/clerkship phase (two years), and clinical experiences phase/internship (two years). Major change interventions in this program include:
Basic sciences phase: (1) organ system approach (horizontal integration); (2) integration of clinical sciences to basic sciences (vertical integration); (3) reduction of factual knowledge with developing core curriculum (core and electives); (4) developing integrated basic science textbooks (student textbook) for managing the organ system courses; (5) early clinical contact.
Introduction to clinical science/preclinical phase: (1) adding” evidence-based medicine “as new content to curriculum and focusing on searching and appraising evidence as part of lifelong learning competency; and (2) adding “communication skills” and professionalism as new contents to the curriculum.
Clinical sciences phase/clerkship: (1) integration of theory to practice in clinical sciences phase and removing pathophysiology phase (vertical integration); (2) integration of pathology, pharmacology, and radiology in clinical sciences rotations; (3) developing integrated study guides for managing integration in clinical science courses; (4) adding “approach to presentation” with more than 130 topics in clinical science courses; (5) longitudinal contact with community health care centers for more than two years and focusing on health promotion and prevention; (6) adding “problem-solving sessions” to clinical course rotations.
Clinical experiences phase/internship: office-based education in primary health care centers.
2. Objectives
This study is a graduate survey on SBGMD curriculum-2004 to identify the strengths and weaknesses of this program and provide data for the curriculum revision.
3. Methods
Cabrera, Weerts, and Zulick identified three approaches to graduate surveys: Outcome approach, engagement and competencies approach, and giving approach. The outcome approach to graduate surveys focuses on characteristics and satisfaction with the institution, as well as engagement and employment since graduation. The engagement and competencies approach to graduate surveys focuses on student experiences in the program, as well as graduate perceptions of competencies attained in the program. The giving approach is focused on the understanding of willingness to support institutions via donations (8). This study is a descriptive cross-sectional survey that focused on the first two approaches. The study was conducted as follows:
3.1. Interviews with Different Stakeholders
We tried to do evaluations based on a participant-oriented approach that seeks to engage broader stakeholders in the evaluation process and respond to their needs. Therefore, we interviewed different stakeholders from managers, faculty members, and students to better understand the reform program, their concerns, interests, and wrote them in the form of the questions that for evaluation they should be responded to (divergent phase).
3.2. Refining Questions and Integrating Them
We reviewed questions, deleted similar questions, merged some questions or integrated them, and then selected the most important questions for answering based on the clients’ interest. Finally, seven key questions remained in the convergent phase, including: (1) What is the employment status of the graduates; (2) Can the graduates undertake their roles in society properly; (3) Can the graduates improve their abilities after graduation and be up-to-date; (4) How have the graduates been affected by reform interventions in basic and clinical science phases; (5) From the viewpoint of the graduates, were the educational facilities and resources enough; (6) How faculty members had affected the graduates; and (7) How were the reform books (student textbooks or study guides) from the graduates’ viewpoint?
3.3. Developing the Questionnaire
We developed a questionnaire based on the evaluation questions. The questionnaire contained 121 questions in seven domains: Demographic information and job status (seven questions), undertaking social responsibility (21 questions), self-improvement competencies (three questions), effects of basic and clinical science interventions (25 questions), adequacy of educational facilities (seven questions), the impact of faculty members (29 questions), and student textbooks and educational guidelines effectiveness (29 questions) with a four-point Likert scale from ‘strongly agree’ to ‘strongly disagree’.
For content validity, the questionnaire was reviewed by an expert panel. The reliability was measured by a test-retest method. The questionnaire was sent to 10 graduates to complete, and after one week, it was completed again by the same individuals, and Cronbach's alpha was 0.76. These respondents were excluded from the final study.
3.4. Data Gathering and Analysis
Sampling was done through a census method, and the questionnaires were sent to the graduates via email or social network in 2017. The inclusion criteria were being graduated under the SBGMD curriculum-2004 from 2011 to 2014 and willingness to participate in the study voluntarily. The exclusion criteria were unwillingness to participate in the evaluation and those with incomplete questionnaires. Initially, telephone calls were made to all graduates to attract more participation, and the importance of the issue was explained.
The scores were summed in each domain, and means and standard deviations were calculated. In this study, 50% of the total score in each domain was determined as a desirable level. For discussion, the authors combined the positive ratings of “agree” and “strongly agree” and negative ratings of “disagree” and “strongly disagree.” Data were analyzed using SPSS version 21 software.
4. Results
In this study, 194 graduates of the SBGMD curriculum-2004 (from 2011 to 2014) participated. The participation rate of the graduates was 51.3% (Appendix 1 in Supplementary File, demonstrating study population and participation rate by year and in total). Among those who did not answer the questionnaire, 14 graduates were asked by telephone again about their general opinion of the SBGMD curriculum-2004. Nine graduates evaluated the program as generally good, two as bad, and three as intermediate.
4.1. Question 1. What Is the Employment Status of the Graduates?
It was observed that 171 graduates were specialists, specialist residents, or were preparing for the medical specialty entrance exam in Iran or abroad (88.6%), and only 22 (11.4%) participants were working as GP (Table 1). On the other hand, 151 (77.9%) graduates found themselves successful in their current positions.
Table 1.
Frequency of Employment Status of SBGMD Curriculum-2004 Graduates (N = 194), 2017
| Employment Status | No. (%) |
|---|---|
| GP in health care centers | 12 (5.6) |
| GP in hospitals | 3 (1.4) |
| GP in private offices | 9 (4.2) |
| GP in private clinics | 17 (7.9) |
| GP in research centers | 2 (0.9) |
| Specialty resident (Iran) | 98 (45.7) |
| Specialty resident (abroad) | 6 (28) |
| Specialists in hospitals | 15 (7) |
| Specialists in research centers | - |
| Specialists in private offices | 2 (0.9) |
| Specialists in private clinics | 1 (0.5) |
| Specialists in health care centers | 1 (0.5) |
| Preparedness for specialty entrance exam (Iran) | 37 (17.3) |
| Preparedness for specialty entrance exam (abroad) | 11 (5.1) |
| Unemployed | - |
| Total | 214a (100 |
aEach participant could choose more than one option.
4.2. Question 2. Can the Graduates Undertake Their Roles in Society Properly?
This question was asked in three parts, including graduate status, graduate career success (mentioned in the previous question), and graduate competencies. All participants evaluated their competencies (self-evaluation) in five areas, including clinical skills, communication skills, patient care, health promotion/prevention, and cognitive skills (reasoning, decision-making, and problem-solving). Each area included sub-competencies as defined by the National Board of GP training for a competent GP, and 70 to 99% of the graduates were competent in each of the five areas (Table 2).
Table 2.
Overall Status of SBGMD Curriculum-2004 Graduates (N= 194) in Five Areas of Competencies, 2017a
| Items | Desirable | Undesirable |
|---|---|---|
| Clinical skills | 192 (99) | 2 (1) |
| Communication skills | 168 (86.6) | 26 (13.4) |
| Patient care (rehabilitation, treatment, diagnosis) | 164 (84.5) | 30 (15.5) |
| Health promotion and prevention competencies | 184 (94.8) | 10 (5.2) |
| Cognitive skills (decision-making, reasoning, and problem-solving skills) | 137 (70.6) | 57 (29.4) |
aValues are expressed as No. (%).
In the clinical skills area, 61.4% of the graduates believed that their knowledge of diseases was not sufficient at the time of graduation. Treatment plan competency (58.8%) and rehabilitation competency (56.7%) in the patient care area, and health advocacy and cooperation with other sectors competencies (51.1%) in the health promotion area were the most defective ones.
4.3. Question 3. Can the Graduates Improve Their Abilities After Graduation and Be Up-to-Date?
To answer this question, two items were asked, and the graduates rated themselves as follows (Appendix 2 in Supplementary File, demonstrating the Graduates’ viewpoints of SBGMD curriculum-2004 (N= 194) on their self-improvement competencies): Self-improvement competency: 71.2% were competent, Evidence-based practice competency: 70% were competent
4.4. Question 4. How Have the Graduates Been Affected by Reform Interventions in Basic and Clinical Science Phases?
To answer this question, 48 items were asked in four parts, as shown in Table 3. Overall, based on the graduates’ viewpoints, the reform interventions were more desirable in the clinical science phase than in the basic science phase (Table 3).
Table 3.
Graduates’ Viewpoints of SBGMD Curriculum-2004 (N = 194) on the Effect of Reform Interventions and Applicability of the Basic and Clinical Science Courses, 2017a
| Desirable | Undesirable | |
|---|---|---|
| The effect of reform interventions in the clinical science phase on graduates’ performance | 178 (91.8) | 16 (8.2) |
| The effect of reform interventions in the basic science phase on graduates’ performance | 170 (87.6) | 24 (12.4) |
| Graduates’ viewpoints on the applicability of basic science courses | 127 (65.5) | 67 (34.5) |
| Graduates’ viewpoints on the applicability of clinical science courses | 179 (92.3) | 15 (7.7) |
aValues are expressed as No. (%).
Also, the results showed a significant difference between the applicability of clinical sciences courses (92.3%) and that of basic sciences courses (65.5%) (P < 0.001). The applicability rates in basic science courses were as follows: Pharmacology courses 91.3%, Anatomy courses 83.4%, Physiology courses 82.5%, Pathology courses 76.3%, Microbiology courses 67.6%, Immunology courses 58.6%, Genetics courses 47.7%, Histology courses 45.4%, Emberiology courses 34.4%, and Biochemistry courses 27%.
In the basic science phase, the” organ system approach” (99%) was more desirable than the “integration of clinical science to basic science courses” (86.6%) (see Appendix 3 in Supplementary File, demonstrating graduates’ viewpoints of SBGMD curriculum-2004 (N = 194) on the effects of basic sciences phase interventions on the graduates’ performance).
In the clinical science phase, the “approach to presentation” was the most effective or desirable intervention, while “communication skills education” and “problem-solving sessions” were the least desirable (Table 4).
Table 4.
Graduates’ Viewpoints of SBGMD Curriculum-2004 (N = 194) on the Effects of Reform Interventions in Clinical Sciences Phase on Graduates’ Performance, 2017a
| Desirable | Undesirable | |
|---|---|---|
| Integration of theory to practice | 154 (79.4) | 40 (20.6) |
| Approach to presentation | 169 (87.1) | 25 (12.9) |
| Professionalism education | 128 (66) | 66 (34) |
| Communication skills education | 118 (60.8) | 76 (39.2) |
| Clinical round and post-round | 140 (72.6) | 53(27.4) |
| Problem-solving sessions | 122 (64.3) | 68 (35.7) |
| Clinical skill lab | 146 (75.6) | 47 (24.4) |
aValues are expressed as No. (%).
Also, the respondents stated that bedside clinical teaching (83%) was more effective than outpatient education (79.4%) and didactic education (74.2%) for the diagnosis and treatment of their future patients (Appendix 4 in Supplementary File, demonstrating graduates’ viewpoints of SBGMD curriculum-2004 (N = 194) on the effects of different clinical education modes on the graduates’ performance).
4.5. Question 5. From the Viewpoints of the Graduates, Were the Educational Facilities and Resources Enough?
Six items were designed to assess this question (Table 5). The results showed that the majority of the participants assessed educational facilities (total 64.4%) and educational hours as desirable (78.4%).
Table 5.
Graduates’ Viewpoints of SBGMD Curriculum-2004 (N= 194) on the Adequacy of Educational Facilities and Teaching Hours, 2017a
| Facilities | Items | |||
|---|---|---|---|---|
| Strongly Agree | Agree | Disagree | Strongly Disagree | |
| The educational facilities in the skill lab were enough for training | 12 (6.3) | 64 (33.3) | 94 (49) | 22 (11.4) |
| The hospital equipment and clinics were adequate for practical training | 10 (5.2) | 68 (35.6) | 93 (48.7) | 20 (10.5) |
| The hospital libraries were enough equipped for clinical training | 7 (3.6) | 83 (43.2) | 80 (41.7) | 22 (11.5) |
| The university library was enough equipped for medical training | 10 (5.2) | 108 (56) | 53 (27.4) | 22 (11.4) |
| Total hours devoted to basic science courses were adequate | 27 (14.1) | 116 (60.4) | 36 (18.7) | 13 (6.8) |
| Total hours devoted to clinical science courses were adequate | 18 (9.4) | 101 (52.3) | 62 (32.1) | 12 (6.2) |
aValues are expressed as No. (%).
4.6. Question 6. How Had Faculty Members Affected the Graduates?
Two items were designed, and the graduates answered them. The results showed that 77.8% of the respondents agreed with the positive effect of clinical faculty members in enhancing their knowledge and skills, and only 45.9% of the respondents believed that the basic faculty members helped to enhance their knowledge and skills (Appendix 5, demonstrating graduates’ viewpoints of SBGMD curriculum-2004 on the effects of faculty members on them).
4.7. Question 7. How Were the Reform Books (Student Textbooks or Study Guides) from the Graduates’ Viewpoint?
For this question, 29 items were designed, and the graduates answered them for each student textbook or study guide, and the results were presented to the relevant department. However, there was a difference between the effect of student textbooks (80.4%) on enhancing students' learning in the basic science phase and study guides in the clinical science phase (60.3%).
5. Discussion
The purpose of this study was to determine the medical graduates’ viewpoints on the big reform in Iran. More than 88% of the respondents (response rate = 5 1.3%) were specialists, specialty residents, or were preparing to enter specialty residency, which indicates they did not accept General Practitioner as a career goal and tended to pass this stage. This finding is very important and may profoundly influence their views on community-oriented (GP-oriented) interventions, and these interventions should be reviewed in the revision of the curriculum. Perhaps looking at the UME program as a transition phase to specialty programs would be a good rationale for rethinking educational strategies in the whole program.
More than 94% of the graduates evaluated themselves as competent in clinical skills and health promotion, which is a strength of the program. But, health advocacy (48.9%), rehabilitation (43.3%), knowledge of diseases (38.6%), and cognitive skills (29.4%) (decision-making, reasoning, and problem-solving skills) showed serious problems and needed more attention in the revision of the curriculum. On the other hand, the defect of treatment plan competency could be due to the interference of interns’ duties by specialty residents in tertiary referral hospitals, which causes to spend more time in indirect patient care and deprives them of learning opportunities (9).
Besides, 95.4% of the graduates reported their self-improvement competency at a desirable level, which is a strong point of the program. In all the interventions, the “organ system approach” (99%) and “approach to presentation” (87.1%) were most favorable for the graduates, and they are the strong points of the program. However, “communication skills education (60.8%), “problem-solving sessions” (64.3%), and “professionalism education” (66%) are the weaknesses of the program and need more attention because of the importance of these topics in the new millennium (2).
Based on the graduates’ viewpoints, bedside teaching was more effective than an outpatient clinic, which could indicate a further emphasis on bedside teaching in the whole program, and it can be a warning sign.
This can be due to training in tertiary referral hospitals, and to comply with new world trends, needs to be more transferred from hospital wards to outpatient clinics (10).
The under-utilization of basic science courses (65.5%) needs more attention and could increase with more integration of clinical science into the basic science phase (more vertical integration) like PBL courses, and this intervention even may cause better performance in clinics (11).
In this study, the worst situation from the graduates’ viewpoints was related to educational resources, with 54.6% desirability, requiring serious attention, especially in skill labs, clinics, and hospital libraries. Fortunately, the Medical School is investing in an equipped clinical skill center that could make up for many of these shortcomings.
The clinical faculty members (77.8%) were more effective than basic faculty members (45.9%) on the graduate’s performance, which could be because the clinical faculty members are physicians and are close to the graduates’ future role. Also, the low applicability of basic science courses may interfere with this result, but further studies on the causes should be undertaken.
In terms of the effectiveness of the reform books in enhancing student learning, respondents believed that student textbooks were more favorable than study guides. It could be because comprehensive test resources (pre-internship exam and residency entrance exam) do not necessarily comply with study guides and conveyed conflicting messages to the students. But, overall, it seems that study guides need to be reviewed and updated.
5.1. Conclusions
In this graduate survey, more than 50% response rate is noticeable, which makes it valuable. Overall, the program was well in many areas, but it also has weaknesses that need to be carefully considered in curriculum revision.