Background:Cardiopulmonary arrest is one of the most critical medical emergencies in the emergency department. Nurses’ knowledge and skills play an essential role in successful cardiopulmonary resuscitation. Therefore, holding in-service training courses is considered vital.
Objectives:This study evaluated the effectiveness of an in-service training course (ISTC) on adult cardiopulmonary resuscitation (CPR) based on the Kirkpatrick model.
Methods:This cross-sectional descriptive study was conducted on 50 emergency department nurses of 5-Azar Hospital in Gorgan, Iran. Training effectiveness was implemented on the four levels of Kirkpatrick's model (including reaction, learning, performance, and findings). The raw data were analyzed by the SPSS version 21 software with descriptive statistics (frequency, mean and standard deviation) and inferential statistics (including Paired t-test and Independent Samples t-test) in a significant level of α = 0.05.
Results:The overall effectiveness of ISTC for adult CPR was reported as 83.23%. The chance of cardiopulmonary resuscitation success increased from 25% in the first six months to 53% in the second six months. In addition, the effectiveness of ISTC was observed in the four levels of the Kirkpatrick model.
Conclusions:Based on the findings, in-service CPR training had a favorable effect on all four levels of the Kirkpatrick model for nurses. The findings of this study significantly increased the efficiency of personnel because evaluating the quality of training courses made them more effective.
Cardiopulmonary arrest is one of the primary and essential reasons for urgent care in the emergency department (1, 2). Every year, about 700,000 people suffer from cardiopulmonary arrest in Europe and the United States (3). Cardiovascular disease is Iran’s leading cause of death, accounting for 46% of all deaths and 20 - 23% of the disease burden (4).
When CPR is performed correctly, quickly, and on time, the patient's chance of survival doubles to three times (5). According to the American Heart Association report, starting CPR in the first 3 - 5 minutes after cardiopulmonary arrest and increasing the chances of survival reduces complications after cardiopulmonary arrest (6, 7). Correct CPR prevents about 90% of cardiac deaths in the emergency room (8). Studies in Iran have reported that the rate of standard CPR is equal to 30 - 50% (4).
When CPR is performed incorrectly, the chance of patients being discharged alive from the hospital is less than 7%, and this issue causes negative consequences for patients and society (9, 10). However, when CPR is principled and standard, it saves patients from death (11). The treatment staff's need for more knowledge and awareness regarding the CPR process is one of the most important causes of CPR failure (12). For this reason, in-service training courses are vital in improving nurses’ knowledge and clinical skills regarding the CPR process (13). Therefore, the CPR training course is the most critical in-service training program for nursing (14). Published reports have shown that the rate of successful CPR in Iran is lower than the standard (15). This problem shows that training alone is not enough, and it is essential to check the effectiveness of training (16).
Educational evaluation is one of the most important programs of any organization, which provides good information on the design and revision of each system to the executors of each training course (17). Effectiveness is achieved when there is a change in the learner's performance in the work environment and the transfer of their learning to the real environment (14). Evaluating educational effectiveness means determining to what extent the training leads to the creation of skills the organization needs in a practical way (18).
There are many models and patterns to determine the effectiveness of training courses, but Kirkpatrick's evaluation model is one of the most important (19, 20). This model evaluates in-service training in four levels: Reaction, learning, performance, and finding (19, 21). At the first level, the reaction is the minimum expectation from a training program. (16, 22). The second level of learning involves assessing learners' mastery of educational goals and pre-and post-tests (18, 23). The third level is performance, which is the learners' use of training in learning environments (19, 22, 24). The finding of the fourth level is the degree of realization of educational goals (25). At this level, the effectiveness of educational programs is evaluated in terms of reducing mortality and infection and increasing patient satisfaction, production, and productivity (22).
However, the evaluation is simple at the Kirkpatrick model's first and second levels in most educational courses in Iran. Few educational programs are comprehensively evaluated this subject. In addition, the educational effects decrease as we approach the third and fourth levels of the model (18). Considering the importance of CPR in reducing death, the CPR training program for medical staff, especially nurses, is regarded as one of the essential job skills (26). The complete mastery of nurses and other workers in correctly applying CPR skills brings patients back to life (14). In addition, the Kirkpatrick model is a comprehensive, simple, and practical model for evaluating clinical courses (15).
Based on the Kirkpatrick model, this study evaluated the effectiveness of an in-service training course (ISTC) on adult cardiopulmonary resuscitation (CPR).
3.1. Study Design
This cross-sectional descriptive study was conducted on 50 emergency department nursing staff of 5th Azar Hospital in Gorgan, Iran, in 2022. The inclusion criteria were a bachelor’s degree in nursing, at least one year of work experience, and not taking sick leave in the six months before and after participating in the desired training course. The title of the in-service training course was based on the latest cardiopulmonary resuscitation guidelines of the American Heart Association (27) and the new guidelines of the Iranian Ministry of Health (24-26, 28). The content presented in the course included the topics of dealing with cardiac arrest patients, basic resuscitation (BLS: Basic life support), advanced resuscitation (ACLS: Advanced cardiac life support), working with an electroshock device, and post-resuscitation care (29).
The training course content was approved by the hospital's emergency medicine specialist, cardiologist, and educational supervisor with the scientific degree of a master's degree in intensive care nursing and a master's degree in internal surgical nursing. Then, the study participants were trained in two groups of 25 people for two days in the implementation phase of this training. A teacher of the training course explained the purpose of the training course before it was implemented to study participants. In addition, the training course teacher emphasized that the pre-test and post-test scores and evaluations aim only to empower the personnel, and all evaluations remain confidential.
3.2. Implementation of the Kirkpatrick Model
The Kirkpatrick model evaluated the training course in four levels: Reaction, learning, performance, and findings. In the first level (reaction), a 12-question questionnaire was used to evaluate the level of the learners to the program, its enjoyment, and the satisfaction level of the participants, which included several factors related to the content, instructor, and facilities, respectively. The participants completed this questionnaire at the end of each course for each group. This questionnaire had a 5-item Likert scale, where "1" was assigned to the "very poor" option and "5" was assigned to the "very good" option. The final score was divided by the total number of questions, and the score for each person was reported as a number between 1 and 5. Finally, the score obtained for each person was calculated based on the number 100 to equalize the scores throughout the research, and a weighting factor equal to "1" was considered for this level.
The second level (learning) was conducted to determine the level of skill and techniques learned during the training course. A questionnaire of 20 questions was designed based on the title and objectives of the training course to evaluate the second level. The content validity of the questionnaire was confirmed using the "content validity index" and "content validity ratio" by three experts in nursing education and two experts in emergency medicine. The value of the "content validity index" and "content validity ratio coefficient" was obtained as 0.8 and 0.99, respectively, which indicates a high level of experts' agreement. The reliability of this questionnaire was evaluated through Cronbach's alpha, and the alpha level for the entire questionnaire was 0.85. Finally, the score obtained for each individual was calculated based on the number 100 to equalize the scores throughout the research, and a weighting factor equal to "2" was considered for this level.
The third level (performance) was conducted to evaluate nurses' behavioral and functional changes. At this stage, the behavioral and functional changes of the nurses in the real and natural environment were reviewed by the supervisors and management of the resuscitation team using a questionnaire. This questionnaire included 14 two-choice questions ("yes" and "no"). The validity of this tool was confirmed by the content method, and its reliability was confirmed by Cronbach's alpha method, the value of which coefficient was equal to 0.78. Finally, the score obtained for each person was calculated based on the number 100 to equalize the scores throughout the research, and a weighting factor equal to "3" was considered for this level.
The fourth level (finding) can be evaluated in four areas: Cost avoidance, saving, benefit, and strategic findings to achieve training findings. This study used a questionnaire to determine the strategic effects so stakeholders could assess how well the training course achieved the intended findings. The officials of the resuscitation team in the hospital completed this questionnaire. This questionnaire included seven five-choice questions that educational supervisors and clinical supervisors completed. The content validity of the questionnaire was confirmed using the "content validity index" and "content validity ratio" by three experts in nursing education and two experts in emergency medicine. The reliability of this questionnaire was evaluated by determining Cronbach's alpha, and the alpha level for the entire questionnaire was 0.75. At this level of the model, the CPR indicators in the six months before the training course were compared with the six months after it to evaluate the effectiveness of the training course.
3.3. Statistical Analysis
The obtained findings were analyzed using SPSS version 21 statistical software using descriptive statistics (including frequency, mean and standard deviation) and inferential statistics (including Paired Sample t-test and Independent Samples t-test) in a significance level of α = 0.05.
The findings of the first level of the Kirkpatrick model (reaction) showed that the satisfaction level of the research participants from the CPR training course was equal to 57.88 ± 3.66, and this satisfaction level based on the number "100" was equivalent to 46 ± 6.11 96 was estimated (Tables 1 and 2). The second-level evaluation (learning) findings showed that the participants' average CPR skill learning score before the intervention was 11.56 ± 2.28 and 14.26 ± 2.21 after the intervention. The findings of the Paired Sample t-test indicated significant differences before and after the intervention (P < 0.01). The post-test score related to CPR skills based on the number "100" was estimated as 71.3 ± 11.05. In addition, the findings of the third level (performance) revealed that the average performance score of the participants was 13.48 ± 0.93, and this value was 96.28 ± 6.65 based on the number "100". The average of the fourth level (effectiveness) was equal to 4.78 ± 1.64, which was obtained based on the number "100" equal to 53.24 ± 1.44 (Table 2). The findings comparing the cardiopulmonary resuscitation status in the first six months with the second six months are presented in Table 3 based on some demographic variables of the patients.
|Content of the training course||19.38 ± 1.25|
|The ability and literacy level of the teacher||19.36 ± 1.2|
|The quantity and quality of the necessary facilities to hold the training course||19.14 ± 1.22|
|The level of overall satisfaction based on the score obtained from the questionnaire||57.88 ± 3.66|
|Overall satisfaction based on the number "100"||96.46 ± 6.11|
The Effectiveness of the Kirkpatrick Model on the Cardiopulmonary Resuscitation Training Course a
|Evaluation Levels||Based on the Score Obtained from the Questionnaire||Based on the Score "100"||Weighting Coefficient|
|First level (reaction)||57.88 ± 3.66||96.46 ± 6.11||1|
|Second level (learning)||14.26 ± 2.21||71.30 ± 11.05||2|
|Third level (performance)||48.58 ± 13.10||95.97 ± 13.10||3|
|Fourth level (results)||4.78 ± 1.64||53.24 ± 1.44||4|
|Overall index||88.23 %|
Comparison of Cardiopulmonary Resuscitation Status in the First Six Months with the Second Six Months
|Variables||The First Six Months||The Second Six Months||P-Value|
|Age of patients||59.91 ± 17.4||59.61 ± 18.3||0.91|
|Recovery time||33.42 ± 17.3||36.8 ± 7.18||0.16|
|Gender of patients||0.4|
|Female||37 (36)||37 (30)|
|Male||66 (64)||74 (70)|
|Result of recovery||0.01|
|Successful||25 (24)||59 (53)|
|Unsuccessful||78 (76)||52 (47)|
The findings indicated the effectiveness of the in-service training course for emergency nursing personnel regarding the cardiopulmonary resuscitation (CPR) process. The chance of the training mentioned above course being successful was estimated to be 88.23%. In addition, participants' satisfaction with the course content, the instructor's teaching style, and the course facilities was relatively high according to their responses on the first level of the model (reaction). The findings of some similar studies in the past are consistent with the present study. Hojjati et al. and Dehghani et al. showed that nurses have a high level of satisfaction with the way of holding the CPR training course (14, 18). Often, this level of evaluation is conducted by a questionnaire about the conditions of holding the course and the instructor's skills, and its findings play an essential role in improving the implementation quality of the CPR training course (25). There are disagreements about evaluating this level because the review of the first level (reaction) does not play an important role in examining educational goals, and mainly this category of reaction is subject to interest and motivation (30). Therefore, for this reason, researchers consider this level (reaction) with a lower coefficient to evaluate the effectiveness of the CPR training course.
The findings at the second level of Patrick's model indicated that the post-test scores increased and the learning rate of the participants in the CPR training course enhanced. The findings of most studies showed that in-service training increases learning and clinical skills, which increases the care and protection of patients (30). Motaarefi et al. reported that the learning rate of nursing students increased significantly before and after the CPR training course (28). Akbari et al. found that the CPR training course increased the learning level of nurses and paramedics in the internal, surgical, and emergency departments (26). Al-Hadid and Suleiman concluded that the average scores after the intervention have a significant difference and increased the knowledge and skills of nurses (31). Hojjati et al. and Dehghani et al. evaluated the effectiveness of in-service training courses for nurses based on Patrick's model, which was also consistent with the findings of the present study in the learning phase (14, 18). Bakhshi-Hajikhajeloo and Sattari also achieved similar findings in examining the effectiveness of brain CPR training workshops in the clinical environment based on the Kirkpatrick model (32).
In the present study, the finding of the third level (performance) of nurses and the fourth level (findings) also indicated the achievement of the majority of the goals of the CPR training course, and in general, the training course was practical. In line with the present study’s findings, Khaledi et al. showed that the pre-organizer training method significantly increased learning and sustainable learning in the training course participants (33). In this regard, Chang et al. investigated the effect of triage start training with face-to-face training on the level of awareness of personnel and indicated that triage training with the face-to-face method could significantly increase awareness in this field. In addition, personnel errors in prioritizing the injured were reduced and increased the ability to perform correct triage (34).
Holding CPR courses for nurses positively increased their efficiency. In general, the findings of this study were consistent with those of Aoki et al.. The findings of the mentioned study showed that implementing the CPR training program in all three levels of the Kirkpatrick model was desirable and had positive effectiveness (35). Omar et al. also confirmed the usefulness of the assessments conducted on middle managers of health in different provinces of Iran based on the Kirkpatrick model (36).
One of the limitations of this research was the lack of a control group for comparison, which can limit the generalizability of the study to some extent. In addition, this study was conducted only on nurses of a teaching hospital, and only the effectiveness of a training course was evaluated. Therefore, it is necessary to be careful in generalizing its findings to other medical centers and training courses.
Based on the findings, the cardiopulmonary resuscitation training course increases nursing skills, knowledge, and efficiency. In addition, based on the findings of this study, it can be said that the course is practical for nurses. The findings showed that the Kirkpatrick model could be a suitable method for evaluating the effectiveness of in-service training in health. Since the practical and clinical skills of nurses are complex activities and the correct implementation of many skills guarantees the life of patients. Therefore, it is suggested to evaluate nurses' learning and functional skills continuously and regularly. In addition, it is recommended to use up-to-date and efficient models in the field of health sciences for proper evaluation.
Shabannia A, Pirasteh A, Jouhari Z. [The effectiveness of cardiopulmonary resuscitation training by mannequin training method and educational video on the awareness of Shahed University staff]. Daneshvar Med. 2021;29(4):33-41. Persian.
Dorri S, Akbari M, Dorri Sedeh M. Kirkpatrick evaluation model for in-service training on cardiopulmonary resuscitation. Iran J Nurs Midwifery Res. 2016;21(5):493-7. [PubMed ID: 27904633]. [PubMed Central ID: PMC5114794]. https://doi.org/10.4103/1735-9066.193396.
Pivac S, Gradisek P, Skela-Savic B. The impact of cardiopulmonary resuscitation (CPR) training on schoolchildren and their CPR knowledge, attitudes toward CPR, and willingness to help others and to perform CPr: Mixed methods research design. BMC Public Health. 2020;20(1):915. [PubMed ID: 32532235]. [PubMed Central ID: PMC7291419]. https://doi.org/10.1186/s12889-020-09072-y.
Khademian Z, Hajinasab Z, Mansouri P. The Effect of Basic CPR Training on Adults' Knowledge and Performance in Rural Areas of Iran: A Quasi-Experimental Study. Open Access Emerg Med. 2020;12:27-34. [PubMed ID: 32110121]. [PubMed Central ID: PMC7042564]. https://doi.org/10.2147/OAEM.S227750.
Y Y, Z F, M N. Cardiopulmonary resuscitation education with serious game on base smart phone and simulation on the attitude of nursing students in Aja University of Medical Sciences. Military Caring Sci. 2018;5(2):95-103. https://doi.org/10.29252/mcs.5.2.95.
Ojaghi Haghighi SH, Shams Vahdati S, Mahmoudie T, Sepehri Majd P, Mirza-Aghazadeh-Attari M. Outcomes of cardiopulmonary resuscitation in the emergency department. J Emergency Practice Trauma. 2017;3(2):49-52. https://doi.org/10.15171/jept.2017.14.
Panchal AR, Bartos JA, Cabañas JG, Donnino MW, Drennan IR, Hirsch KG, et al. Part 3: Adult basic and advanced life support: 2020 American Heart Association guidelines for cardiopulmonary resuscitation and emergency cardiovascular care. Circulation. 2020;142(16_Suppl_2):S366-468.
Habibli T, Ghezeljeh TN, Haghani S. The effect of simulation-based education on nursing studentsâ knowledge and performance of adult basic cardiopulmonary resuscitation: A randomized clinical trial. Nursing Practice Today. 2020;7(2):87-96.
Abdollahi H, Tayebeh P, Mazlum R, Malekzadeh J, Farsi M, Janati F. [Effect of capnography feedback during cpr on return of spontaneous circulation]. Medical J Mashhad Univ Med Sci. 2018;61(1):816-24. Persian.
Kavosi A. [Effect of continuing education competency-based program for emergency nurses on rate CPR successful]. Quarterly J Nurs Manag. 2015;4(2):0. Persian.
Guetterman TC, Kellenberg JE, Krein SL, Harrod M, Lehrich JL, Iwashyna TJ, et al. Nursing roles for in-hospital cardiac arrest response: Higher versus lower performing hospitals. BMJ Qual Saf. 2019;28(11):916-24. [PubMed ID: 31420410]. https://doi.org/10.1136/bmjqs-2019-009487.
Birkun A, Gautam A, Trunkwala F. Global prevalence of cardiopulmonary resuscitation training among the general public: A scoping review. Clin Exp Emerg Med. 2021;8(4):255-67. [PubMed ID: 35000353]. [PubMed Central ID: PMC8743682]. https://doi.org/10.15441/ceem.21.066.
Mahdavi R, Askarpour A, Heydari B, Morovati A, Delshad Z, Maghsoodloo E. [The Effect of training based on extended parallel process model on adherence to medication regimen among diabetic elderly]. Inter J Med Inv. 2022;11(4):100-8. Persian.
Dehghani MH, Tafti AD, Raadabadi M. [The effectiveness of training courses on cardiopulmonary resuscitation (CPR) based on Kirkpatrick Model]. J Nurs Edu. 2019;8(3). Persian.
Sadeghi E, Bahmani H, Khorshidi S. [Evaluation of cardiopulmonary resuscitation status and its related factors in Shahid Motahhari hospital of Marvdasht: A descriptive cross-sectional study]. Pars Jahrom Univ Med Sci. 2020;18(1):23-8. Persian. https://doi.org/10.52547/jmj.18.1.23.
Hamemoradi M, Khorasani A, FathiAjargah K. [Evaluation of the effectiveness of the on-the-job training courses in NIGC based Kirkpatrick, Phillips and CIPP models]. Strategic Studies Petroleum energy Industry. 2014;5(19):103-30. Persian.
Matsuoka Y, Goto R, Atsumi T, Morimura N, Nagao K, Tahara Y, et al. Cost-effectiveness of extracorporeal cardiopulmonary resuscitation for out-of-hospital cardiac arrest: A multi-centre prospective cohort study. Resuscitation. 2020;157:32-8. [PubMed ID: 33080369]. https://doi.org/10.1016/j.resuscitation.2020.10.009.
Hojjati H, Mehralizadeh Y, Farhadirad H, Alostany S, Aghamolaei M. [Assessing the effectiveness of training outcome based on Kirkpatrick model: Case study]. Quarterly J Nurs Manage. 2013;2(3):35-42. Persian.
Ahanchian M, Sharafi S, Vafaee M, Hajiabadi F. [Evaluate the Effectiveness of internship program in nursing student using Kirkpatrick's model]. Res Med Edu. 2017;9(1):17-9. Persian. https://doi.org/10.18869/acadpub.rme.9.1.17.
Lee H, Song Y. Kirkpatrick model evaluation of accelerated second-degree nursing programs: A scoping review. J Nurs Educ. 2021;60(5):265-71. [PubMed ID: 34039135]. https://doi.org/10.3928/01484834-20210420-05.
Badran AS, Keraa K, Farghaly MM. Applying the Kirkpatrick model to evaluate dental students' experience of learning about antibiotics use and resistance. Eur J Dent Educ. 2022;26(4):756-66. [PubMed ID: 34939280]. https://doi.org/10.1111/eje.12758.
Shayan S, Nowroozi Rad N. [Evaluation of the effectiveness of staff in-service training system, Tehran Taleghani Hospital with Kirickpatrik approach]. J Med Spiritual Cultivation. 2019;28(2):10-23. Persian.
Chellaiyan VG, Suliankatchi RA. Health research methodology workshop: Evaluation with the Kirkpatrick model. Natl Med J India. 2019;32(2):100-2. [PubMed ID: 31939408]. https://doi.org/10.4103/0970-258X.275352.
Ghorbandoost R, Zeinabadi HR, Shabani Shafiabadi M, Mohammadi Z. [Evaluation of in-service training course of nurses and midwives (Neonatal resuscitation) using kirkpatrick's model]. Res Med Edu. 2020;12(3):4-11. Persian.
Farmad SA, Yousefian F. Evaluation of the Effectiveness of Virtual CPR Training in Midwives of Shahid Hasheminezhad Hospital in Mashhad. Med Edu J/Āmūzish-i Pizishkī. 2022;10(2).
Akbari M, Dorri S, Mahvar T. [The effectiveness of in-service training on cardiopulmonary resuscitation: Report of first and second levels of Kirkpatrick's model]. Develop Strategies Med Edu. 2016;3(1):67-72. Persian.
Smidt A, Balandin S, Sigafoos J, Reed VA. The Kirkpatrick model: A useful tool for evaluating training outcomes. J Intellect Dev Disabil. 2009;34(3):266-74. [PubMed ID: 19681007]. https://doi.org/10.1080/13668250903093125.
Motaarefi H, Sakhaei S, Ashrafi M, Zinalpoor S, Homed Gholizadgougjehyaran H. [Effect of teaching standard precautions based on Kirkpatrick’s model on the knowledge and practice of nursing students]. Qom Univ Med Sci J. 2022;16(6):452-61. Persian.
Cahapay M. Kirkpatrick model: Its limitations as used in higher education evaluation. Inter J Assessment Tools Edu. 2021;8(1):135-44. https://doi.org/10.21449/ijate.856143.
Siaulys MM, da Cunha LB, Torloni MR, Kondo MM. Obstetric emergency simulation training course: Experience of a private-public partnership in Brazil. Reprod Health. 2019;16(1):24. [PubMed ID: 30813967]. [PubMed Central ID: PMC6391815]. https://doi.org/10.1186/s12978-019-0689-6.
Al Hadid L, Suleiman KH. Effect of boost simulated session on CPR competency among nursing students: A pilot study. J Educ Practice. 2012;3(16):186-93.
Bakhshi-Hajikhajeloo S, Sattari S. [Effectiveness evaluation of cardiopulmonary cerebral resuscitation workshop in the clinical atmosphere]. Cardiovasc Nurs J. 2017;6(3):16-22. Persian.
Khaledi S, Moridi G, Shafieian M, Gharibi F. [The effect of a combination of lecture and participatory on the stable learning of nursing students]. J Nurs Edu. 2010;20(4):76-80. Persian.
Chang S, Kwon E, Kwon Y, Kwon H. The effects of simulation training for new graduate critical care nurses on knowledge, self-efficacy, and performance ability of emergency situations at intensive care unit. Kor J Adult Nurs. 2010:375-83.
Aoki T, Asada Y, Hiroe T, Tomizawa Y. Board 105 - Program Innovations Abstract Effectiveness of American Style CPR Training for Japanese Medical Students (Submission #1494). Simulation in Healthcare: J Soci Simulation Healthcare. 2013;8(6). https://doi.org/10.1097/01.Sih.0000441370.61737.26.
Omar M, Gerein N, Tarin E, Butcher C, Pearson S, Heidari G. Training evaluation: A case study of training Iranian health managers. Hum Resour Health. 2009;7:20. [PubMed ID: 19265528]. [PubMed Central ID: PMC2654422]. https://doi.org/10.1186/1478-4491-7-20.