Characteristics of the Internal Evaluations of the Educational Groups in Kermanshah University of Medical Sciences, Iran

1. Background

Communities need universities for progress and development, and these centers are the most valuable institutions in this field. Due to the growing number of students and universities and the subsequent economic, social, and educational issues, it is essential to improve the quality of education and examine the extent to which it has been achieved so far (1). Policymakers must pay special attention to the quantitative and qualitative improvements in the higher education system. The purpose of quantitative and qualitative improvement is to enhance educational performance, improve the higher education system, increase a sense of responsibility in institutions toward each other, and ensure the adequacy of graduates and quality control (2).

Intragroup evaluation is a process through which faculty members, students of different levels, and university officials examine the strengths and weaknesses of their department in various fields in an attempt to overcome the obstacles and problems in education. Research, patient treatment, and other factors that hinder achieving the primary goals of the field could provide appropriate feedback to colleagues and officials for effective decision-making and addressing these shortcomings (3).

Evaluation is a means to increasing the efficiency of higher education and a mechanism for certification, reasonable fund allocation, informing stakeholders, ensuring the achievement of goals, and continuous improvement. In Iran, the evaluation and accreditation program of medical universities was approved in the third period of the five-year development plan of Iran. The first plan of the Iranian Ministry of Health and Medical Education (MHME) is goal-oriented internal evaluation, followed by the external evaluation of various departments of medical universities based on the internal evaluation. In the process of internal evaluation, items such as improving the quality of the educational system, self-regulation, and participation of faculty members in group activities in line with the university system and society’s needs lay the foundation for increasing the quality of the educational system (4). Educational evaluation is a formal and purposeful activity, which is designed and implemented to determine the quality, efficacy, and value of an educational program or process (5). Internal evaluation is performed by the educational unit to clarify the quality and quantity of the educational and research status in the form of self-evaluation. In general, the purpose of internal evaluation is to determine the current status and its distance from the desired state (6). Attention to evaluation and feedback is a clear sign of attention to the quality of higher education, followed by systematic planning to improve the situation (7).

Recognizing the extent to which the goals of medical universities are achieved and knowledge of the shortcomings and strengths (current status) are essential tools that help decision-makers, policymakers, and planners of medical universities improve the applied methods, achieve goals, and increase efficiency. Moreover, dynamics and growth in these universities require constant monitoring and evaluation (8). The importance of external evaluation in the management of higher education is undeniable. Universities seek to continuously improve the quality of their programs and educational, research, and specialized services system by applying various solutions. International and national experiences show that the external evaluation process (especially at the departmental level) is an effective mechanism for ensuring academic quality. On the other hand, external evaluation is used as a general term referring to all forms of quality review, review, and validation. External evaluation is the process used by external experts to evaluate and ensure the quality of a program/institution. Undoubtedly, external evaluation should be based on internal evaluation, so that an internal evaluation report would be available before implementing the process (9).

In the past two decades, several studies have been focused on the performance and internal evaluation of departments in medical universities in different regions of Iran, including Kermanshah.

2. Objectives

Considering that the reports of these intragroup evaluations have not been examined quantitatively and formally so far, the present study aimed to review the intra-group evaluations in 2012 - 2018 as an introduction to the additional studies of these evaluations to provide effective solutions for improving the quantity and quality of university departments.

3. Methods

This cross-sectional, descriptive study was conducted to review the internal evaluations of various medical departments of Kermanshah University of Medical Sciences (KUMS), Iran during 2012 - 2018. The sample population was the group assessments, and sampling was census by selecting all the evaluation reports. Documented information was also included in the final report of these evaluations in a checklist containing 40 items (based on seven factors of internal evaluation, including (1) objectives, organizational position, management and organization; (2) faculty members; (3) students; (4) learning-teaching strategies; (5) educational facilities and equipment; (6) thesis, study opportunities, and seminars; and (7) graduates), which was extracted in the evaluation for summarization.

Data analysis was performed in SPSS version 21 using descriptive statistics (frequency and percentage for the qualitative variables and mean for quantitative variables) and bar charts.

The study protocol was approved by the Medical Educational Development Center (EDC), Research Council, and Ethics Committee of KUMS (code: IR.KUMS.REC.1399.360).

4. Results

During the six years of the study, 22 evaluations were performed regarding the departments of KUMS, and 11 cases (50%) were related to the medical school (Figure 1). Departments such as ophthalmology, neurology, pharmaceutics, and medical emergencies were evaluated twice. The largest number of evaluations was in 2015 (n = 9), and the smallest number was in 2012 and 2013 (n = 1). Most of these evaluations were performed within six months (n = 13; 59.1%), with the minimum and maximum duration determined to be three months and one year, respectively. In addition, the number of the pages of the final report was within the range of 36 - 184 pages.

Frequency of Evaluations Performed in Medical Schools of KUMS

Figure 1.

Frequency of Evaluations Performed in Medical Schools of KUMS

In 81.8% of the reports (n = 18), a briefing/introduction session was held for the group members at the beginning of the evaluation, and in 86.4% of the cases, an intragroup evaluation committee was formed. In 95.5% of the evaluations (n = 21), the mission and vision of the department were specified, and in 100% of the cases, the areas of evaluation and the criteria for assessing and desirability of the current status accordance with each of the criteria were determined and scored.

In 68.2% of the cases, the method of data collection was internal evaluation through a questionnaire, while an interview was performed in only one case (Figure 2). The evaluation methods included workshops, pamphlets, booklets, books, and experts in the field of evaluation, which were exploited to acquaint the faculty members with the evaluation process in 77.3% of the cases. In 95.5% of the cases, the method of department manager selection was based on specific instructions and procedures, and in 100% of the cases, there was a specific job description of the department manager, as well as a specific plan for the recruitment of new faculty members. In more than 80% of the cases, there was a specific program for the scientific development, skills, encouragement, and evaluation of the faculty members within the department. In 100% of the documented research activities of the faculty members, there was a specific and codified program for conferences, seminars, and scientific conferences.

Frequency of Performed Evaluations Based on Data Collection Method

Figure 2.

Frequency of Performed Evaluations Based on Data Collection Method

In 100% of the cases, the teaching methods used within the department were specific and documented, and there were cases of lesson plans for theoretical units. Lesson plans were also available in 86.4% of the cases for practical units and 68.2% of the cases for skill units of the clinical departments. In 100% of the cases, the lesson plan was delivered to the students, and in 90.9% of the cases, it was uploaded to the department’s website. In more than 90% of the cases, the welfare facilities for the faculty members and students were mentioned, and in 100% of the cases, documents were confirming the evaluation of the students by the faculty members, as well as the evaluation of the department manager by the faculty members.

In 50% of the cases, the departments provided rooms for the faculty members and students and had an independent class for the group. However, only 40% had a well-documented program to assess the students’ academic decline/achievement within the departments. In terms of graduates, 50% of the departments provided documents regarding communication with the graduates, feedback from the graduates, information on the graduates’ professional transcript, continuing the graduates’ education, and publishing the graduates’ researches. In 45% of the cases, no information was reported about the characteristics of the faculty members of the department, and in 35% of the cases, only the academic rank of the faculty members was mentioned.

In 68.2% of the reports, the problems and limitations of the department were mentioned, and in 77.3% of the cases, suggestions were made to the officials to solve the problems and limitations of the department, reflect these problems, and initiate follow-up. In 77.3% of the reports, content/resources were produced by the members of the department, and in 95.5% of the cases, there was a report on the expenses and budget of the department. Only 27.3% of the experts were recruited in the group evaluation committee, and in most of the cases (72.7%), educational experts were absent from the team. In 54.5% of the cases, student representatives were recruited in the group evaluation committee. In 72.7% of the cases (n = 16), all the faculty members of the department participated in the evaluation. In most of the cases (72.7%), no specific program was available regarding the cooperation of the experts within the group.

5. Discussion

According to the results of the present study, KUMS evaluations were performed correctly, quantitatively, and formally, and our findings indicated that they could achieve multiple academic goals. In more than 80% of the performed evaluations, a briefing/introduction session had been implemented at the beginning of the evaluation for the group members to rate the formation of an intragroup evaluation committee, the mission and vision of the group, and determining the evaluation areas, evaluation criteria, and the desirability of the current status appropriate to each criterion.

Our findings demonstrated that the educational goals of the departments were specified, and the method of group manager selection was based on specific instructions and procedures, as well as a description of the duties of the group manager and a specific plan for the recruitment of new faculty members. A specific program for scientific development included outlines on skills, encouragement, and evaluation of the activities of the faculty members within the department, documentation of the research activities of the faculty members, a specific and codified program for symposiums, scientific seminars, and conferences. The teaching methods in the group were specific and documented, and there were lesson plans for theoretical units and practical units, which were delivered to the students or uploaded on the department website. Furthermore, the welfare facilities provided to the faculty members and students had been listed on the reports. The student’s evaluations of the professors and evaluation of the group manager by the professors were also documented, and a report was also available on the expenses and budget of the departments.

The reported rate in the final reports of internal evaluations for the following cases was less than 50%: rooms had been provided to the faculty members and students, and independent classes had been considered for the department, as well as a special documentary to assess the academic decline/progress of the students within the groups and the graduates. The reports also included the profile of the faculty members and the recruitment of teaching experts and student representatives in the group evaluation committee.

The results of the intragroup evaluation study of the dermatology department of KUMS indicated that this department could cover six of seven factors (1) goals, organizational status, and management and organization; (2) faculty members; (3) Students; (4) teaching-learning strategies; (5) educational facilities/equipment; and (6) study opportunities and seminars) in an acceptable state. In terms of the seventh factor (7 graduates), this factor was considered to be in a moderate state, which is consistent with the results of the present study (3).

According to Anjam Shoa et al. (10), the seven factors mentioned in methods and the above paragraph along with organizational status/department management, faculty members, educational/research facilities and equipment, and graduates of the department had a favorable status. On the other hand, the factors of students, teaching-learning strategies, and training courses and curricula were only moderately favorable. Furthermore, the three categories of the input, process, and output factors of the business management department were reported to be favorable. As a result, the general state of the business management group of the Faculty of Management of the University of Tehran (Iran) was reported to be favorable, which is consistent with our findings (10).

The findings regarding the external evaluation of the Department of Restorative and Dental Materials of the School of Dentistry of Babol University of Medical Sciences (Iran) indicated that regarding the factor of faculty members, the individual personality component of the faculty members had a favorable status, while the components of teaching quality, assessment skills, and interpersonal communication were unfavorable. Moreover, the results of the goals and mission factor indicated that the educational and research components, space, equipment, and facilities of information technology, goals and mission, evaluation, and organization had a favorable state in terms of value burden. Summarizing the data obtained from the evaluation of the equipment and facilities of the Department of Restorative and Dental Materials demonstrated the favorable status of the department in terms of clinical equipment, electronic services, and technology, while the state of the library, physical space, and welfare services was relatively favorable. Since the mentioned study involved an external evaluation, their findings could be inconsistent with the present study in different aspects (11).

In another study, Sepahi et al. (12) investigated the status of outpatient education from the perspective of the trainees and clinical interns of KUMS. To this end, they conducted a cross-sectional, descriptive study in the academic year 2012 - 2013, and data were collected using a researcher-made questionnaire with confirmed validity and reliability. Their findings showed that the students’ views on various aspects of clinical education indicated an unfavorable status in this regard (12). In another study, Monfared and Safi (13) compared the performance appraisal and ranking of various departments in a public university, reporting that the departments with the same inputs were inefficient in some of the evaluation components.

In the study by Dehghani et al. (14), the educational departments of Shahid Beheshti teaching hospital in Kashan (Iran) were evaluated, and the views of trainees and interns about the status of the round and grand round in most of the departments indicated a relatively favorable status in this regard. The results of the aforementioned studies are inconsistent with our findings since they mainly involved an external evaluation. Therefore, it could be concluded that internal evaluation and external evaluation yield different outcomes.

External evaluation is essential within universities. External evaluation is performed by an evaluator outside the evaluated organization in two manners of in-university and out-of-university. The difference between these two formats is that in the first mode, the committee consists of specialists from inside the university, while they are selected from outside the university in the second mode. At this stage, the goal is to review the results of the internal evaluation by those who have been selected from outside the department and belonged to the training group. The purpose of this group is to determine the validity of the results of the internal evaluation so that they could report their observations and judgment based on each of the factors, criteria, and indicators used in the internal evaluation. The results of such external evaluations may not differ greatly from the results of internal evaluations or they may differ significantly. The optimal validation results are obtained with coherence between the performances of the two evaluation groups. Therefore, there is an urgent need to form an external evaluation team within KUMS.

5.1. Conclusions

According to the results, quantitative and formal evaluations were performed correctly, and the outcomes briefly indicated that multiple goals could be achieved. In some cases (e.g., physical space and issues related to students/graduates), the reported rates were not considered appropriate. In addition, the results of the internal evaluations differed from the external evaluations and group problems in several cases (e.g., group organizational chart, group facilities). Therefore, there is an urgent need to form an external evaluation team within KUMS or assist experts in the evaluation and the review of the organizational issues within the university departments.