1. Background
Evaluation is essential in the qualitative improvement of services delivered by an organization (1). Most public sector health expenditure is spent in hospitals (2-4), and the majority of these costs occur due to improper performance of hospitals (1). Thus, evaluation of performance and health system reform are essential for the proper management of the hospital (5).
The purpose of health system reform in many low and middle-income countries is the delivery of universal health care services (6). The four factors toward health system reform include increasing costs of health care, rising expectations of the people, limited financial resources and spending that almost all countries are confronted with them (7). According to the Islamic Republic of Iran’s 20-Year Vision Plan and Iran's General Health Policy, the healthcare reform plan (8) began on May 5, 2014, with three approaches to protection and financial support of patients, creation of people's accessibility to health services and improvement in the quality of services (9). In the vision of this plan, raising the level of public satisfaction and reducing the costs of health services in public hospitals are considered as the major priorities of the program (10). Basically, the purpose of implementing a healthcare reform plan is to improve the health of people, reduce the amount of payments out of pocket and promote the health indicators of community (11, 12).
Evaluation is performed on the basis of predetermined standards. Iran Hospitals Evaluation Criteria are codified under Article 8 of the Law on the Establishment of the Ministry of Health and implemented by the Hospitals Evaluation Workshop in the Department of Treatment. The values achieved are compared to the standard, and the favorable, moderate, and unfavorable values of each of the hospital performance indicators are communicated by the Ministry of Health (1).
| Type of Indicators | Favorable | Moderate | Unfavorable |
|---|---|---|---|
| Percentage of bed count to bed constructed (%) | 75 to 80 | 60 to 74 | Less than 60 |
| Bed occupancy rate (%) | More than 70 | 60 to 70 | Less than 60 |
| Bed performance ratio | More than 24 | 17 to 24 | Less than 17 |
| Bed turnover interval (day) | Less than 2 | 2 to 3 | More than 3 |
| Admission ratio per bed | More than 24 | 17 to 24 | Less than 17 |
| Average length of stay (day) | Less than 3.5 | 3.5 to 4 | More than 4 |
| Surgery to operating room bed ratio (surgery per day) | 4 | 2 to 4 | Less than 2 |
| Died patients to hospitalized (admitted) patient’s ratio | Less than 2 | 2 to 3 | More than 3 |
hospital indices reflect hospital performance and should be collected at specified intervals so that the possibility of comparison between them with national standards, different regions, and other hospitals be provided (13). Various indices are used to measure hospital performance (1, 3, 11, 16) the most important of which are the three indices of Bed Occupancy Rate, Bed Turnover Rate, and hospital average length of stay (ALOS) (1, 3, 11).
2. Objectives
The study aimed to evaluate the performance in five domains and compared these indices before and after the implementation of the healthcare reform plan in Iran and with the standard of the Ministry of Health. Besides, the status of the hospitals under study was investigated using the Pabon Lasso model.
3. Methods
This was a cross-sectional descriptive-analytical study which was performed in 2017. The study was conducted in seven educational hospitals (three general hospitals and four super-specialty hospitals) located in the city of Kermanshah. Thus, the study sample included hospitals that the healthcare reform plan was implemented in them that all of which were selected through census sampling.
The study resources were statistical data of the hospitals. The data were related to hospital performance indices including 1- indicators of beds and hospital stay (10 indicators), 2- indicators of patients (4 indicators), 3- indicators of surgical procedures (8 indicators), 4- indicators of mortality (3 indicators), and 5- indicators of para-clinical services (12 indicators). As well as, the status of hospitals was evaluated using the Pabon Lasso model, and the comparison of indicators was also performed with Iranian standards.
The data were collected using a researcher-made data extraction form. The form consisted of five main sections based on the indicators studied. The validity of the form was confirmed by 7 individuals (including 5 people of health information management and 2 people of biostatistics). To compare the collected data and determine the status of hospital performance, the list of Iran's MoH standard indicators was used (Table 1).
The healthcare reform plan was implemented in mid-May 2014 in educational hospitals of Iranian universities. Thus, to calculate the indicators, the data for May were not included. Therefore, the data of one year before the implementation of the plan (May 2013 to March 2014) and one year after the implementation of the plan (June 2014 to May 2015) were collected. The study was approved by the ethics code IR.KUMS.9697015 and permission to obtain data was received from the Department of Research and Technology of the university. The data were collected by referral of the researcher to the Statistics Unit of the Deputy of Treatment and were recorded in data extraction forms. Data analyzed using SPSS 20, and the indicators were compared. Besides, the indicators were compared with the standard performance indicators of the Iran hospitals evaluation criteria are codified under Article 8 of the Law on the Establishment of the Ministry of Health. The status of the studied hospitals for one year before and after the implementation of the plan was evaluated and compared using the Pabon Lasso model.
4. Results
In this study 37 indices in five areas were surveyed. Comparing performance indicators of bed and hospital stay before and after the healthcare reform plan indicates that, out of ten indicators of this domain, nine indicators have improved. The bed constructed had been increased for two hospitals that the average bed count increased as well. Percentage of bed count to bed constructed declined. As indicated in Table 2, the indicators of bed turnover interval (TOI) and the ALOS after implementation of the plan decreased.
Table 2.Comparison of Bed and Hospital Stay Indices
| Indices | Average of Indices | P-Value | |
|---|---|---|---|
| Before Health Reform Plan | After Health Reform Plan | ||
| Bed constructed | 2621.71 | 2816.86 | 0.21 |
| Average of bed count | 2603.71 | 2648.43 | 0.18 |
| Percentage of bed count to bed constructed | 98.51 | 93.78 | 0.31 |
| Total bed count day | 7.94 | 8.08 | 0.18 |
| Bed count day | 5.31 | 5.93 | 0.10 |
| Percentage of bed occupancy | 62.58 | 68.01 | 0.02 |
| Admission ratio per bed | 7.55 | 8.11 | 0.03 |
| Bed turnover rate | 7.59 | 8.22 | 0.06 |
| Bed turnover interval | 1.80 | 1.33 | 0.01 |
| Average length of stay (ALOS) | 3.57 | 3.37 | 0.38 |
Indicators related to patient represent that the number of admitted (132968 to 145823) and discharged (131361 to 145477) patients and patients staying less than 24 hours (24381 to 24922) enhanced. And so, the total number of mortality also increased (2053 to 2173).
Indicators of surgery showed that seven indicators increased and two indicators have declined, although decreasing cesarean rate has been one of the objectives of the healthcare reform plan that about 13 percent of which have been realized (Table 3).
Table 3.Comparison of Surgical Indices a
| Indices | Before Health Reform Plan | After Health Reform Plan | P-Value |
|---|---|---|---|
| Outpatient surgeries | 3958.86 ± 8998.67 | 4605 ± 10928.34 | 0.41 |
| Inpatient surgery | |||
| Elective | 3968.14 ± 4693.04 | 4565.57 ± 5116.57 | 0.17 |
| Emergency | 5985.43 ± 12102.28 | 6288.86 ± 11233.22 | 0.6 |
| Normal delivery | 668.14 ± 1285.26 | 764.29 ± 1500.80 | 0.29 |
| Cesarean section | 650.14 ± 1152.57 | 669 ± 1175.18 | 0.32 |
| Cesarean section rate | 14.60 ± 26.81 | 1.53 ± 2.77 | 0.2 |
| Ratio of surgery to operating room bed | 1.44 ± 3582.20 | 1.01 ± 108.51 | 0.35 |
| Ratio of surgery to the number of inpatient admissions | 0.39 ± 0.33 | 0.41 ± 0.36 | 0.61 |
| Ratio of outpatient surgery to total patient | 0.09 ± 0.20 | 0.1 ± 0.23 | 0.44 |
aValues are expressed as mean ± SD.
Mortality indices after the healthcare reform plan showed that the mean of all indicators after implementation of the healthcare reform plan had decreased. The changes in these indicators for outpatient department services increased. In the inpatient department, it declined only for endoscopic services. Providing the pacemaker services was decreased. As illustrated in Table 4, changes in para-clinical service delivery indices were significant only for laser therapy with P-value = 0.01, Admitted Lab Test with P-value = 0.04, and ratio of tests performed to inpatients with P-value = 0.01.
Table 4.Comparison of Clinical Services Indices a
| Clinical Services | Before Health Reform Plan | After Health Reform Plan | P-Value |
|---|---|---|---|
| Endoscopy | |||
| Inpatient | 243.14 ± 326.33 | 226.14 ± 414.66 | 0.84 |
| Outpatient | 328.14 ± 428.8 | 483.43 ± 628.63 | 0.23 |
| Spirometry | |||
| Outpatient | 123.5 ± 19.09 | 131.5 ± 127.98 | 0.93 |
| Laser therapy | |||
| Outpatient | 511 ± 52.32 | 1364.5 ± 81.31 | 0.01 |
| Electromyography | |||
| Inpatient | 124 ± 159.8 | 188.5 ± 256.68 | 0.51 |
| Outpatient | 1137 ± 386.08 | 1207 ± 453.96 | 0.38 |
| Sonography | |||
| Inpatient | 4928.8 ± 5772.63 | 5710.6 ± 7068.67 | 048 |
| Outpatient | 5727.2 ± 6927.47 | 8081.6 ± 8761.43 | 0.11 |
| peace maker | |||
| Temporary | 82 ± 108.89 | 14.5 ± 13.43 | 0.5 |
| Permanent | 60.5 ± 85.56 | 10.5 ± 14.84 | 0.5 |
| Plastering | |||
| Outpatient | 327.71 ± 867.05 | 889.43 ± 2353.2 | 0.35 |
| Admitted lab test | |||
| Inpatient | 49944.57 ± 47140.23 | 59693.71 ± 56632.21 | 0.04 |
| Outpatient | 18506 ± 13633.25 | 22042.14 ± 14685.61 | 0.17 |
| Performed lab test | |||
| Inpatient | 163674.14 ± 213944.5 | 394688.43 ± 385776.11 | 0.14 |
| Outpatient | 98676.57 ± 91911.04 | 190727.14 ± 122553.02 | 0.11 |
| Ratio of performed lab test to inpatients | |||
| Inpatient | 5.63 ± 2.75 | 7.15 ± 3.49 | 0.01 |
| Admitted pathology | |||
| Inpatient | 2200.25 ± 3106.6 | 2657.75 ± 3412.65 | 0.11 |
| Outpatient | 1192.25 ± 719.65 | 1914.75 ± 1025.97 | 0.05 |
| Performed pathology | |||
| Inpatient | 1269 ± 2488.54 | 1519.57 ± 2799.98 | 0.13 |
| Outpatient | 686.71 ± 817.03 | 1097.14 ± 1257.21 | 0.07 |
aValues are expressed as mean ± SD.
Comparing hospital performance indicators with the standards of the Ministry of Health showed that both Bed Turnover Rate and Admission Ratio per bed before and after the healthcare reform plan were in an unfavorable condition Table 5. The percentage of bed count to bed constructed for the six hospitals before the healthcare reform plan was in a favorable condition, and one hospital was in a moderate condition. This situation remained constant for all centers after the plan.
Table 5.Comparison of Performance Indices Based on Iranian Ministry of Health in Hospital Related to Kermanshah University of Medical Sciences
| Hospital 1 | Hospital 2 | Hospital 3 | Hospital 4 | Hospital 5 | Hospital 6 | Hospital 7 | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Value | Status | Value | Status | Value | Status | Value | Status | Value | Status | Value | Status | Value | Status | |
| Percentage of bed count to bed constructed | ||||||||||||||
| Before | 107.48 | F | 100 | F | 87.3 | F | 107.04 | F | 112.12 | F | 106.33 | F | 69.33 | M |
| After | 108.26 | F | 100 | F | 87.3 | F | 77.42 | F | 106.1 | F | 104.77 | F | 72.77 | M |
| Percentage of bed occupancy | ||||||||||||||
| Before | 60.36 | M | 33.71 | UF | 51.83 | UF | 68.76 | M | 71.44 | F | 71.48 | F | 80.48 | F |
| After | 71.24 | F | 41.51 | UF | 50.36 | UF | 70.81 | F | 82.06 | F | 76.08 | F | 83.5 | F |
| Bed turnover rate | ||||||||||||||
| Before | 6.76 | UF | 6.85 | UF | 6.93 | UF | 10.42 | UF | 5.67 | UF | 14.34 | UF | 2.2 | UF |
| After | 7.64 | UF | 8.44 | UF | 7.77 | UF | 9.68 | UF | 6.72 | UF | 14.73 | UF | 2.61 | UF |
| Bed turnover interval | ||||||||||||||
| Before | 1.79 | F | 2.95 | M | 2.12 | M | 0.91 | F | 1.54 | F | 0.61 | F | 2.71 | M |
| After | 1.15 | F | 2.11 | M | 1.95 | F | 0.92 | F | 0.79 | F | 0.5 | F | 1.93 | F |
| Admission ratio per bed | ||||||||||||||
| Before | 6.77 | UF | 6.85 | UF | 6.95 | UF | 10.01 | UF | 5.71 | UF | 14.35 | UF | 2.21 | UF |
| After | 7.69 | UF | 8.14 | UF | 7.49 | UF | 9.56 | UF | 6.64 | UF | 14.87 | UF | 2.48 | UF |
| Average length of stay (ALOS) | ||||||||||||||
| Before | 2.72 | F | 1.5 | F | 2.28 | F | 2.01 | F | 3.84 | M | 1.52 | F | 11.17 | UF |
| After | 2.85 | F | 1.5 | F | 1.98 | F | 2.23 | F | 3.75 | M | 1.58 | F | 9.75 | UF |
Abbreviations: F, favorable; UF, unfavorable; M, medium.
Performance indices analysis of hospitals before and after the health reform plan using the Pabon Lasso model showed that, before the implementation of the health reform plan, Percentage of Bed Occupancy in three hospitals was more than 70 percent (favorable status), two hospitals had between 60 - 70 percent (moderate status), and two hospitals had less than 60 percent (unfavorable status) Table 6. After the implementation of the health reform plan, the index was in favorable status in five hospitals and unfavorable status in two hospitals. The highest bed occupancy rate in the 2014 and 205 were 80.48 and 83.50 percent, respectively.
Table 6.Performance Indices of Kermanshah University of Medical Sciences Hospitals Before and After Implementation of Health Reform Plan
| Hospital Code | Before Health Reform Plan | After Health Reform Plan | ||||
|---|---|---|---|---|---|---|
| Percentage of Bed Occupancy | Average Length of Stay (ALOS) | Bed Turnover Rate | Percentage of Bed Occupancy | Average Length of Stay (ALOS) | Bed Turnover Rate | |
| 1 | 51.83 | 2.65 | 6.93 | 50.36 | 2.08 | 7.77 |
| 2 | 71.44 | 3.84 | 5.67 | 82.62 | 3.75 | 6.72 |
| 3 | 68.76 | 2.01 | 10.42 | 70.81 | 2.23 | 9.68 |
| 4 | 60.36 | 2.72 | 6.76 | 71.24 | 2.85 | 7.64 |
| 5 | 71.48 | 1.52 | 14.34 | 76.08 | 1.58 | 14.73 |
| 6 | 83.50 | 11.17 | 2.20 | 80.48 | 9.75 | 2.61 |
| 7 | 41.51 | 1.50 | 6.85 | 33.71 | 1.50 | 8.44 |
In 2014 (before the implementation of health reform plan), Pabon Lasso graph illustrated that three hospitals were in the first district, two hospitals in the second district, and two hospitals in the fourth district (Figure 1).
Figure 1.
Status of Kermanshah university hospitals on Pabon Lasso graph before implementation of health reform plan in 2014.
In 2015 (after the implementation of health reform plan), one hospital was in the first district, one hospital in the second district, two hospitals in the third district, and three hospitals in the fourth district (Figure 2).
Figure 2.
Status of Kermanshah university hospitals on Pabon Lasso graph after implementation of health reform plan in 2015.
5. Discussion
in this study, 38 indicators were evaluated in five domains. In the indicators associated with the bed and hospital stay, after the implementation of the healthcare reform plan, 195 beds licensed (bed capacity) were added. The rate of indicators of average bed count, total bed day, and bed occupancy day was increased. Adding the number of beds constructed to hospitals was the reason for the increase in these indicators. The study conducted after the implementation of the healthcare reform plan in Turkey represented that the number of beds count had increased by 18 percent (18).
The percentage of bed occupancy by a 6 percent increase had a significant change (P = 0.02). After the implementation of the healthcare reform plan, investigation of the performance indicators at the universities of Bushehr, Hamedan, and Shiraz demonstrated that the percentage of bed occupancy of the plan for all increased (11, 19-21). The bed turnover interval with -0.47 and P-value = 0.01 and the average length of stay also declined. The reason for this decrease was the high referral of patients and the increase in the number of hospitalizations, which corresponded to a similar study conducted by Rezaei (11). A study carried out in Bushehr revealed that there was an increase in bed turnover interval for two hospitals, but this increase was not significant (21). As well as, a study conducted in Isfahan indicated that the reduction of bed turnover interval after the healthcare reform plan was significant (19).
In the year after the implementation of the healthcare reform plan, the number of inpatients increased to 12855 and discharged patients to 14166. Given the greater number of inpatients and discharged patients, the mortality rate also increased. The study of Ghasemzadeh et al indicated that the average number of inpatients and discharged patients, as well as the mortality rate after the healthcare reform plan, was increase (19). A study of examining Turkey's health system reform demonstrated that the number of inpatient increased by 30.3 percent (18). The mean of all surgeries (outpatient, inpatient) after the implementation of the healthcare reform plan had increased. The reason for this increase is a reduction in patient’ out-of-pocket payments and much referral. There was no statistically significant difference in the increase of these indicators, which was in line with the results of the study conducted by Sulku in Turkey (18) and Faridfar et al. (8). While the results of the study carried out by Bastani et al. showed that the rate of surgery after the healthcare reform plan was associated with a significant increase (20), and study of Ghasemzadeh et al. represented that the rate of surgical operations after the plan was decreased in Isfahan (19).
The cesarean section rate decreased by about 13 percent after the implementation of the reform plan, but this difference was not significant compared to the previous year, which was consistent with the results of the study of Rezaei et al. (11). One of the objectives of the reform plan was to reduce the cesarean section rate and to promote normal vaginal delivery increased (5); nevertheless, the results indicated that the reform plan could be effective in reducing the cesarean section rate during one year of implementing this study.
Despite the increase in the number of inpatients, outpatients and the number of surgeries, the gross and net mortality rate decreased by 0.18. The mortality rate in the study conducted by Ghasemzadeh et al. and the net mortality rate in the study of Sulku (18) increased after the healthcare reform plan. Due to the lack of data related to infant and maternal mortality rates, there was no possibility of their analysis in this study. Thirteen indicators related to para-clinical services for inpatients and outpatients were assessed separately. For the radiotherapy services provided, no data were recorded. In the cases of para-clinical services indicators, except the two indices of the mean number of patients admitted to endoscopy and the pacemaker, the rate of all other indicators increased. In a similar study carried out by Faridfar et al., the number of para-clinical services after implementation of the healthcare reform plan increased (8).
Comparing performance indicators of university hospitals in Kermanshah with the standards of the Ministry of Health indicated that the indicators of bed performance ratio and admission ratio per bed before and after the healthcare reform plan were in unfavorable condition for all hospitals. The patients' average length of stay (ALOS) before and after the reform plan was favorable in five centers and moderate in one center. For the psychiatric center, this indicator before and after the reform plan had an unfavorable condition that the nature of patients admitted to this hospital requires long-term hospitalization (22), so it is not possible to comment with certainty about the unfavorable condition of the average length of stay at the center. Comparing the performance indicators of Kermanshah hospitals in two years before the implementation of the reform plan demonstrated that the average length of stay in two centers was unfavorable (14).
The results of the study for these two centers indicated that the average length of stay before the plan was unfavorable for these centers and changed to moderate condition after the implementation of the reform plan in a hospital and still remained unfavorable for the psychiatric center. The reason for the unfavorable condition of this indicator in the psychiatric center is the nature of its patients. A comparison of the status of hospitals using the Pabon Lasso model was conducted with three performance indicators of bed occupancy rate, the average length of stay and bed turnover rate. Investigation of the status of these indicators with the standards of the Ministry of Health provided results that indicated that three hospitals were in the first position, two hospitals in the second position, and two hospitals in the fourth position before the implementation of the healthcare reform plan. After the implementation of the healthcare reform plan, one hospital was in the first position, one hospital in the second position, two hospitals in the third position, and three hospitals in the fourth position, representing the change in the status of hospital indicators and the positive impact of the healthcare reform plan on the indicators.
5.1. Conclusion
Evaluating the hospital performance indicators before and after the implementation of the healthcare reform plan, as well as comparing the indicators with the standards of the Ministry of Health, demonstrated that the implementation of the reform plan has had a positive impact on most of the indicators and has improved them and mortality rates have all declined. To conduct this study, there were limitations to the analysis of some indicators. The data related to some indicators were not provided to the Statistics Unit of the Deputy of Treatment. For some indicators, the data were not recorded. In some cases, the structure of the data was in a way that there was no possibility of comparison.
