Abstract
Background:
As one of the most important components of curriculums, the educational environment provides learners with a forum in which they can learn communication skills and other abilities such as critical thinking and clinical problem-solving. To improve this environment and consequently upgrade medical education, the current environment must be evaluated.Objectives:
In a quest to adapt a valid and reliable instrument for this purpose, this study aimed to assess the reliability and validity of the Persian translation of Dutch Residency Educational Climate test (D-RECT) to measure the quality of the educational environment in the main clinical wards of Shiraz University of Medical Sciences.Methods:
In a cross-sectional study, a forward Persian translation of the D-RECT questionnaire was translated and back-translated by two bilingual expert panels, and its reliability was determined in a pilot study. Then, 100 residents working at internal medicine, surgery, pediatrics, and gynecology wards were selected randomly to fill out the questionnaire. Data were analyzed using SPSS version 21 software.Results:
The Persian translation of D-RECT had acceptable validity, reliability, internal consistency, and reproducibility. The reliability of the questionnaire was confirmed by Cronbach’s α of 0.95. Convergent validity was 0.61 to 0.90. Among the wards evaluated in this study, the educational environment was significantly better in the pediatric ward than in the other wards.Conclusions:
The Persian translation of D-RECT validated in the studied departments seems to be an effective, valid, and reliable instrument for evaluating and comparing the quality of the current educational environment for residents in Iran. This evaluation can help improve training.Keywords
1. Background
Nowadays, not only the medical training boards but also society and residents themselves demand a high-quality residency training program (1). Educational climates demonstrate the forum in which the residents thrive and therefore exhibit the various aspects of a training program. Every learning climate is shaped based on interactions between learners, educators, patients, and other health care providers, as well as tools such as electronic devices, charts, and rounds. In other words, it represents the participants’ and educators’ common belief regarding the learning atmosphere, status, and approaches in each department. A method to assess the quality of educational programs is to assess learning climates. Learning climates tell us about the settings and context that residents participate in. Learning climates are built through relationships between students and other healthcare workers (2). However, as a theoretical construct, the learning climate cannot be measured by itself and should be evaluated through measurable key attributes such as learners’ daily experiences. Based on a qualitative study done by Boor (3) concerning an effective learning climate, residents consider their daily work as an opportunity for further learning rather than just a requirement to fulfill. In such a climate, the residents themselves are in charge of educational decision-making, which can satisfy their individual needs. In their opinion, another indispensable part of an ideal climate is the constructive interaction among faculties, peers, and other healthcare staff (3). The development of an evaluation instrument with a high quality of psychometric properties is not an easy task. Many widely accepted instruments for educational environment evaluation need improvement (4-7). Based on a study on surgical residency teaching, although a student-centered method of residency teaching is better than a teacher-centered method, further studies are needed to assess other aspects of residency training environment (8).
The Dutch Residency Educational Climate test (D-RECT) has several advantages compared to other instruments, such as providing more straight-forward answers and evaluating a broader spectrum of learning climates and more aspects of residents’ jobs (2). A study of the validity and reliability of the German version of the D-RECT questionnaire showed that the present replication study with the D-RECT German showed structural differences concerning factorial validity, underpinning the need for further validation studies (9).
2. Objectives
In a quest to adapt a valid and reliable instrument for educational environment evaluation, this study attempted to assess the reliability and validity of the Persian translation of the D-RECT in measuring the quality of the educational environment at the main clinical wards of Shiraz University of Medical Sciences. Our study is unique since, to the best of our knowledge, not only the Persian version of the instrument has not been validated in an Iranian clinical ward setting but also this study is the first to measure the educational environment in Iranian main clinical wards based on the D-RECT.
3. Methods
In this study, 100 residents were randomly chosen from four different clinical wards by using a random selection table. The wards included surgery, internal medicine, pediatrics, and gynecology wards. These four main departments were selected because a major bulk of residents are working in these wards. The residency education in Iran in these major wards lasts four years. In the first and second years of education, the residents are observers and named as junior residents. In the third and fourth years, they are senior residents. In each hospital department where residency education is delivered, one specialist is the “attending tutor” and responsible for residents’ training and assessments. The training program consists of rotations in general hospitals affiliated to the Shiraz University of Medical Sciences and some ambulatory outpatient clinics. The integration of health care delivery and the medical education system in Iran provides a unique opportunity for residents to use the healthcare system for improving education (10).
The original learning climate questionnaire was extracted from a qualitative study done by Boor et al. (2), which consisted of 50 questions in 11 dimensions, including three questions on supervision, eight questions on coaching and assessment, three questions on feedback, three questions on collaboration among peers, four questions on teamwork, three questions on the professional relationship among attendings, four questions on if work is adapted to residents’ competence, eight questions on attendings’ roles, four questions on formal education, six questions on the role of the specialty tutor, and four questions on patient sign-out. The dimensions’ aspects and items were discussed in a bilingual expert panel consisting of medical education experts, medical doctors, and specialized faculty members. We invited seven medical educationalists, seven residents, and seven attending tutors for the expert panel. After the initial revising of the forward translation by the expert panel, the Persian version was back-translated by an independent translator and returned to our expert group for face and content validity. For determining the validity, we used the Content Validity index (CVI) and content validity ratio (CVR). For determining the CVI, experts in the panel were asked to rate the translation of the items of D-RECT questionnaire concerning clarity and relevance on a four-point ordinal scale (1 = not relevant, 2 = somewhat relevant, 3 = quite relevant, 4 = highly relevant). For determining the CVR, experts needed to score each item of the translated questionnaire from 1 to 3, including “not necessary”, “useful but not essential”, and “essential”, respectively (11).
The reliability of the D-RECT domains was measured using Cronbach’s alpha. The reliability was considered to be adequate if the Cronbach’s alpha coefficient was at least 0.7. Furthermore, appropriate statistical analyses, including minimum and maximum scores, mean ± standard deviation (SD), items’ total correlation, and Cronbach’s alpha, were used to determine the items’ properties. The questionnaire was paper-based, and the researcher asked residents to fill it out anonymously. Written informed consent was obtained from all residents. Residents participated in the study voluntarily, and their names were not mentioned in the scripts.
4. Results
The study enrolled 100 residents from four departments, including surgery, pediatrics, gynecology, and internal medicine. The sample included 47 males, 53 females, 37 single residents, and 63 married residents. The CVI and CVR were more than 0.8 in all items of the questionnaire, and the experts approved all the items. The native residents (52%) were more than non-native residents (48%). The questionnaire had an overall Cronbach’s alpha coefficient of 0.958, with dimensions 8 and 11 having the most agreement and dimension 4 the least agreement (Cronbach’s alpha of 0.894 and 0.729, respectively).
The Pearson correlation coefficient showed good correlations between the dimensions of the questionnaire (Table 1). The evaluation of the correlation between the dimensions of the questionnaire showed that dimensions 11 and 7 had the strongest correlation, followed by dimensions 11 and 9 (Table 2).
Correlation Between the Dimensions of the Questionnaire and Its Items
| Dimension | Items | Item with Minimum Correlation | Minimum Correlation Coefficient | Item with Maximum Correlation | Maximum Correlation Coefficient |
|---|---|---|---|---|---|
| 1 | 1 - 3 | 1 | 0.734 | 2 | 0.849 |
| 2 | 4 - 11 | 5 | 0.691 | 6 | 0.755 |
| 3 | 12 - 14 | 12 | 0.660 | 14 | 0.904 |
| 4 | 15 - 18 | 18 | 0.692 | 17 | 0.806 |
| 5 | 19 - 21 | 20 | 0.614 | 20 | 0.756 |
| 6 | 22 - 24 | 23 | 0.624 | 24 | 0.744 |
| 7 | 25 - 28 | 27 | 0.703 | 25 | 0.853 |
| 8 | 29 - 36 | 31 | 0.618 | 33 | 0.853 |
| 9 | 37 - 40 | 40 | 0.763 | 38 | 0.844 |
| 10 | 41 - 46 | 41 | 0.595 | 42 | 0.798 |
| 11 | 47 - 50 | 50 | 0.780 | 48 | 0.858 |
Correlation Between the Dimensions of the Questionnaire
| Dimensions | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 1 | |||||||||
| 2 | 0.649 | |||||||||
| 3 | 0.388 | 0.478 | ||||||||
| 4 | 0.522 | 0.544 | 0.499 | |||||||
| 5 | 0.485 | 0.539 | 0.340 | 0.538 | ||||||
| 6 | 0.455 | 0.674 | 0.456 | 0.664 | 0.702 | |||||
| 7 | 0.447 | 0.601 | 0.382 | 0.555 | 0.598 | 0.684 | ||||
| 8 | 0.598 | 0.749 | 0.472 | 0.499 | 0.604 | 0.643 | 0.701 | |||
| 9 | 0.532 | 0.589 | 0.437 | 0.561 | 0.638 | 0.672 | 0.704 | 0.655 | ||
| 10 | 0.584 | 0.680 | 0.558 | 0.509 | 0.647 | 0.492 | 0.677 | 0.638 | 0.632 | |
| 11 | 0.618 | 0.694 | 0.333 | 0.431 | 0.610 | 0.573 | 0.759 | 0.649 | 0.751 | 0.621 |
The residents’ gender had no statistically significant relationship with dimensions (P < 0.05). The total mean score was 146 for the residents of all departments. The maximum and minimum mean scores were 162 (pediatrics ward) and 128 (gynecology ward), respectively. The mean score was 147 for both internal medicine and surgery departments.
5. Discussion
Based on the results of this study, we developed an overall valid and reliable Persian translation of the D-RECT questionnaire to use for measuring the quality of the educational environment in Shiraz, Iran. The different aspects of the Persian questionnaire had positive correlations, and the scores were very close together, suggesting an optimal correlation between the different dimensions of this study. Convergent reliability showed the maximum convergence of a dimension with its questions, indicating that the questionnaire had good convergence reliability. The differential correlation between dimensions and other questions also suggested that the validity of this questionnaire was good. To our knowledge, the Persian version of this instrument has never been validated and used for the assessment of Iranian educational environments. Since this is the first study conducted based on D-RECT in Iran, there are no similar studies to be compared with our study. Therefore, we compared our results with studies conducted based on other adapted instruments for educational climate evaluation. The Greek translation of the Dundee ready educational environment measure (DREEM) questionnaire had Cronbach’s alpha of 0.9 (12). There was only one study about the reliability of the D-RECT questionnaire in the German language that revealed Cronbach’s alpha between 0.57 and 0.85 (13).
Among the wards assessed in this study, the educational environment of the pediatric ward had the highest score, which agrees with the results of a study done based on the DREEM questionnaire in Iran, in which the pediatric ward achieved the best score in the dimension of professors (14).
The result of a study on the educational environment in residency education in Pakistan based on the postgraduate hospital educational environment measures showed that the highest score was noted in the neurology department, followed by ENT, pediatric, and internal medicine departments. The lowest score was observed for the Anesthesia Department (15). A comparative study of the emergency residency program environment in the USA and Saudi Arabia showed that the scores were significantly higher in the USA residency programs. In subscales, social support was not different between the two countries, but teaching was better in the USA residency programs. There were no significant differences in the residency training years (16). Another study of the educational environment in the urology department in Saudi Arabia showed that urology residents rated the educational environment as less than satisfactory. Also, some areas of the health care delivery system were doing better than other areas (17).
The results of the questionnaire survey did not show significant differences in the different dimensions of the questionnaire that demonstrated the same understanding of the educational environment among men and women. In a study that evaluated students learning environment in Sweden, the total score of 145 out of 200 was achieved that showed a favorable environment (18). In a study in 2008, Arabshahi evaluated the perspective of the learning environment by the PHEEM questionnaire and showed a total score of 99 out of 200 (14).
The strength of the present study is the use of the generally acknowledged and well investigated D-RECT to assess the learning environment of residents (5, 19). In addition, research in the field of educational environment is of priority in medical education in the Eastern Mediterranean Region and Iran (20). There were some limitations to this study. The first limitation was the small sample size and the inclusion of only four major residency education departments in our study. This study was done using the 50-item D-RECT questionnaire, and it was not possible for us to simultaneously use the 35-item D-RECT tool (21). Another limitation is that the D-RECT focuses on paramount issues such as “supervision”, “coaching and assessment”, and “feedback”. In addition, the D-RECT includes less obvious themes such as “professional relationships between attendings” and “patient handover”. Also, there was no opportunity for the evaluation of all residents in clinical wards and the assessment of the relationship between a suitable learning environment and the correct performance of residents.
5.1. Conclusions
The present study evaluated a multidimensional questionnaire for the assessment of the learning environment and determined the reliability and validity of its Persian translation. Also, the strengthens and weaknesses of the wards” managers in the four main wards were evaluated that allows them to support their educational environments and encourage the residents’ learning. The results of this study also helped us improve the relationship between the wards and the success pattern of one ward for a good educational environment to use for another ward.
Acknowledgements
References
-
1.
Genn JM. AMEE medical education guide no. 23 (part 2): Curriculum, environment, climate, quality and change in medical education - a unifying perspective. Med Teach. 2001;23(5):445-54. [PubMed ID: 12098364]. https://doi.org/10.1080/01421590120075661.
-
2.
Boor K, Van Der Vleuten C, Teunissen P, Scherpbier A, Scheele F. Development and analysis of D-RECT, an instrument measuring residents' learning climate. Med Teach. 2011;33(10):820-7. [PubMed ID: 21355691]. https://doi.org/10.3109/0142159X.2010.541533.
-
3.
Boor KB. The clinical learning climate. Amsterdam, Netherlands: VU Medical Center; 2009.
-
4.
Bartram D, Foster J, Lindley PA, Brown AJ, Nixon S. Learning climate questionnaire (LCQ): Background and technical information. Oxford: Employment Service and Newland Park Associates Limited; 1993.
-
5.
Pinnock R, Welch P, Taylor-Evans H, Quirk F. Using the DRECT to assess the intern learning environment in Australia. Med Teach. 2013;35(8):699. [PubMed ID: 23837553]. https://doi.org/10.3109/0142159X.2013.786175.
-
6.
Ashkanasy NM, Jackson CRA. Organizational culture and climate. Handbook of industrial, work & organizational psychology - volume 2: Organizational psychology. Sage Knowledge; 2001. p. 398-415. https://doi.org/10.4135/9781848608368.n21.
-
7.
Roff S, McAleer S, Harden RM, Al-Qahtani M, Ahmed AU, Deza H, et al. Development and validation of the dundee ready education environment measure (DREEM). Med Teach. 2009;19(4):295-9. https://doi.org/10.3109/01421599709034208.
-
8.
Khan JS. Evaluation of the educational environment of postgraduate surgical teaching. J Ayub Med Coll Abbottabad. 2008;20(3):104-7. [PubMed ID: 19610531].
-
9.
Lblher P, Zupanic M, Ostermann T. The questionnaire D-RECT German: Adaptation and test theoretical properties of an instrument for evaluation of the learning climate in medical specialist training. GMS J Med Educ. 2015;32(5):1-7. [PubMed ID: 26604997]. https://doi.org/10.3205/zma000997.
-
10.
Salehi A, Harris N, Lotfi F, Hashemi N, Kojouri J, Amini M. Reform in medical and health sciences educational system: A Delphi study of faculty members' views at Shiraz University of Medical Sciences. East Mediterr Health J. 2014;20(3):151-61. [PubMed ID: 24950072].
-
11.
Zamanzadeh V, Ghahramanian A, Rassouli M, Abbaszadeh A, Alavi-Majd H, Nikanfar AR. Design and implementation content validity study: Development of an instrument for measuring patient-centered communication. J Caring Sci. 2015;4(2):165-78. [PubMed ID: 26161370]. [PubMed Central ID: PMC4484991]. https://doi.org/10.15171/jcs.2015.017.
-
12.
Kanashiro J, McAleer S, Roff S. Assessing the educational environment in the operating room-a measure of resident perception at one Canadian institution. Surgery. 2006;139(2):150-8. [PubMed ID: 16455322]. https://doi.org/10.1016/j.surg.2005.07.005.
-
13.
Dimoliatis ID, Vasilaki E, Anastassopoulos P, Ioannidis JP, Roff S. Validation of the Greek translation of the dundee ready education environment measure (DREEM). Educ Health (Abingdon). 2010;23(1):348. [PubMed ID: 20589604].
-
14.
Soltani Arabshahi K, Kouhpayezade J, Sobuti B. The educational environment of main clinical wards in educational hospitals affiliated to Iran university of medical sciences: Learners' viewpoints based on DREEM model. Iran J Med Educ. 2008;8(1):43-50.
-
15.
Sandhu A, Liaqat N, Waheed K, Ejaz S, Khanum A, Butt A, et al. Evaluation of educational environment for postgraduate residents using post graduate hospital educational environment measure. J Pak Med Assoc. 2018;68(5):790-2. [PubMed ID: 29885185].
-
16.
Aalam A, Zocchi M, Alyami K, Shalabi A, Bakhsh A, Alsufyani A, et al. Perceptions of emergency medicine residents on the quality of residency training in the United States and Saudi Arabia. World J Emerg Med. 2018;9(1):5-12. [PubMed ID: 29290889]. [PubMed Central ID: PMC5717376]. https://doi.org/10.5847/wjem.j.1920-8642.2018.01.001.
-
17.
Binsaleh S, Babaeer A, Alkhayal A, Madbouly K. Evaluation of the learning environment of urology residency training using the postgraduate hospital educational environment measure inventory. Adv Med Educ Pract. 2015;6:271-7. [PubMed ID: 25897272]. [PubMed Central ID: PMC4396647]. https://doi.org/10.2147/AMEP.S81133.
-
18.
Edgren G, Haffling AC, Jakobsson U, McAleer S, Danielsen N. Comparing the educational environment (as measured by DREEM) at two different stages of curriculum reform. Med Teach. 2010;32(6):e233-8. [PubMed ID: 20515368]. https://doi.org/10.3109/01421591003706282.
-
19.
Bennett D, Dornan T, Bergin C, Horgan M. Postgraduate training in Ireland: Expectations and experience. Ir J Med Sci. 2014;183(4):611-20. [PubMed ID: 24390312]. https://doi.org/10.1007/s11845-013-1060-5.
-
20.
Amini M, Kojuri J, Lotfi F, Karimian Z, Abadi AS. Research priorities in medical education in the Eastern Mediterranean Region. East Mediterr Health J. 2012;18(7):687-92. [PubMed ID: 22891514].
-
21.
Silkens ME, Smirnova A, Stalmeijer RE, Arah OA, Scherpbier AJ, Van Der Vleuten CP, et al. Revisiting the D-RECT tool: Validation of an instrument measuring residents' learning climate perceptions. Med Teach. 2016;38(5):476-81. [PubMed ID: 26172348]. https://doi.org/10.3109/0142159X.2015.1060300.