Abstract
Background:
The accurate retrieval of cancer data in health information systems depends on the correct coding and classification of data. Thus, it is essential to provide continuous training for clinical coders and ensure the effectiveness of this training to achieve high-quality data.Objectives:
The present study aimed at evaluating the effectiveness of the training workshop on ICD-10 cancer coding guidelines for clinical coders.Methods:
The present study was performed to evaluate the effectiveness of coding guidelines training regarding the second chapter of ICD-10 (2016 ed.). Forty-five clinical coders have participated in the study. A researcher-made questionnaire was provided for guidelines data gathering, with the scoring system of Likert (0 to 5 points) at the reaction level, and by and pre-and post-test questionnaires at the learning level. The descriptive statistics were applied on the reaction level and the paired sample t-test was performed on the reaction level data.Results:
The results of descriptive statistics revealed that the learners had 84.4% of the average satisfaction with the training courses The paired-samples test indicated that, at the learning level, a significant difference (P = 0.000) existed between the mean pre-and post-test scores.Conclusions:
Based on Kirkpatrick’s model, the training program resulted in satisfaction and improved the clinical coders skills regarding cancer coding. It is recommended that health information managers hold effective training courses to enhance the coders’ knowledge and skills.Keywords
Training Clinical Coding ICD-10 Kirkpatrick's Model Evaluation
1. Background
Today, non-communicable diseases (NCD) are the cause of the majority of diseases and mortality worldwide. Meanwhile, cancer is the main cause of mortality and the most important obstacle to increasing life expectancy in all countries (1). It is a serious threat to health and the second cause of mortality around the world (2). Based on the estimations of the World Health Organization (WHO), in 2015, cancer was the first or second main cause of mortality before the age of 70 years in 91 countries out of the 172 countries of the world.
The incidence of cancer and the resulting mortality is rapidly increasing globally (1). In its 2020 report, the WHO noted that in 2018, 18.1 million people worldwide had cancer, and 9.6 million people lost their lives due to this disease. By 2040, these values will be almost doubled. Cancer is the cause of about 30% of all the cases of mortality due to NCD in adults aged 30 to 69 years (3). Based on the study by Kazem Zendehdel in Iran, about 110,000 cases of cancer occurred in 2018, of whom about 56,000 died (4).
The heavy costs of medical interventions and services for cancer treatment and the costs of hospitalization, outpatient visits, and medications are the consequences of cancer. Moreover, indirect costs such as the loss of productivity due to the disease and early mortality can be regarded as complications of cancer. To these, psychological and social problems resulting from cancer, including the pain and suffering caused by the disease and its treatment, can be added (5).
Mutuma noted that cancer diagnosis has wide-ranging economic and social consequences for the person, family, and society (6).
Controlling cancer aiming to reduce its prevalence, complications, , and mortality, carrying out systematic evidence-based interventions to prevent, early diagnosis and treatments and alleviatory care may lead to quality-of-life improvement among patients (7, 8).
Effective planning for cancer control requires accurate data, including reliable registries for cancer, monitoring, and evaluation programs for quality assurance (9).
According to Bray et al., the quality of data and evaluation of the data in cancer registries should have features of comparability, validity, timeliness, and completeness (10). Furthermore, the WHO published a report in 2016 on cancer prevention and control and emphasized that effective policies for this purpose should be designed based on accurate data (9). Cancer registry data help describe the burden and etiology of cancer, evaluate primary and secondary prevention effects, and perform health service planning (11).
The achievement of these objectives needs the use of information classification and coding standards. International Statistical Classification of Diseases and Related Health Problems,10th Revision (ICD-10) is among the most vital epidemiological tools for monitoring the incidence and prevalence of specific diseases and other health-related problems (12).
The retrieval of information from databases enhances policy-makers’ understanding of the incidence and prevalence of diseases and enables them to implement measures to reduce and track diseases and allocate resources. This will not be realized unless the information is accurately coded and classified (13).
High-quality data are specific, accurate, and comprehensive diagnostic codes that are important both in research and in decision-making (14).
The use of coding instructions improves the accuracy and quality of cancer registry data (15). Wilson et al. stated that an understanding of the completeness of codes and the quality of clinical coding is essential to survival analysis in cancer registries (16). Coding errors prevent the productivity and effective performance and management of hospitals and can potentially lead to inaccurate national statistics about the incidence rate of the disease (17). The quality of coded data depends on two factors: first, to what extent the healthcare providers document the diagnoses and procedures accurately and completely, and second, how accurately and consistently the documents are coded by clinical coders (18). Therefore, attention to high-quality coding is of utmost importance. Still, there are numerous obstacles to high-quality coding: (1) coders limited in their abilities for adding, modifying, or interpreting medical documents, (2) incompleteness of documents, (3) inconsistencies in medical records, (4) the use of different terms for describing clinical diagnoses, and (5) a communication gap between coders and doctors (14). WHO emphasizes that clinical coders need knowledge of medical terminology, legal aspects, health information, and health data standards, and training is an inseparable component of health information (12).
Professional coding training is an essential way for reducing code inconsistencies (19).
Concerning the role of coders in assigning an accurate code, and since the lack of attention to the quality of coding and coding errors leads to the inaccurate classification of diseases, trusting data with unspecified or poor classification quality will have risks for both healthcare providers and managers in planning and epidemiological and medical research (13). This highlights the significance of continuous training and ensuring the effectiveness of this training on accurate cancer classification and coding and, therefore, the achievement of high-quality data in cancer registries.
Coder training provides general and specialized knowledge and skills for using clinical coding standards at a national level (20).
2. Objectiives
Accordingly, the present study aimed at evaluating the effectiveness of a training workshop on ICD-10 cancer coding guidelines for clinical coders.
3. Methods
This is an applied evaluation study. The statistical population comprised the coders working in hospitals affiliated with Shahid Beheshti University of Medical Sciences, Tehran Province (n = 45). The Ethics Committee of Shahid Beheshti University of Medical Sciences approved the study (IR.SBMU.RETECH.REC.1400.108). Kirkpatrick’s evaluation model was adopted to evaluate whether the training program was effective in meeting the needs of the learners (21, 22).
In the present model, the evaluation process consisted of the levels of reaction, learning, behavior, and results. According to the definition of Kirkpatrick, reaction level refers to the learner’s satisfaction. In the learning level, the knowledge or capability gained after the training is assessed. In the behavior level, the quantity and the nature of alteration in the behavior of participants following the training course is measured. At the results level, the degree to which the organization’s objectives of holding the training course is achieved (23, 24).
This study used levels of reaction and learning from Kirkpatrick’s model. At the reaction level, a researcher-made questionnaire was provided to the participants regarding the components of satisfaction in Kirkpatrick’s model. Each questionnaire consisted of 5 close-ended questions to assess the satisfaction of learners using Likert scores including completely disagree, somewhat disagree, neither agree not disagree, somewhat agree and completely agree demonstrating with 1 to 5 respectively. The content validity of the questionnaire was assessed, and its reliability was checked via Cronbach's alpha (α = 0.96).
A pre-test post-test design was used to evaluate the extent to which the coders learned the cancer coding guidlines.
A questionnaire including 10 multiple-choice items and 7 open-ended questiones was given to the participants before and after the workshop.
The content validity of the questionnaire was assessed, and its reliability (0.95) was examined via the test-retest method. As the data distribution was normally based on the Kolmogorov-Smirnov test, the pre- and post-test mean scores were compared via a paired samples t-test. The data were analyzed in SPSS 22, and the significance level of 0.05 was set for the paired samples t-test.
4. Results
The results are divided into levels of reaction and learning.
Reaction level (the learners’ satisfaction level): Based on Table 1, 84.4% of the participants reported that the training program improved their comprehension and awareness of cancer coding. In terms of the teaching method, 80% of the participants were provided the materials by sufficient instructors. In addition, 100% of the learners stated that the material was correlated to their duties as clinical coders. Overall, 93.3% of the participants were satisfied with the training course, and 91.1% of them would suggest this program to other coders.
The Frequentness Distribution of the Participants’ Satisfaction with Effectiveness of Training Workshop on ICD-10 Coding Guidelines of Cancer a
| Criterion | Strongly Disagree | Somewhat Disagree | Neither Agree nor Disagree | Somewhat Agree | Strongly Agree | Total Percentage |
|---|---|---|---|---|---|---|
| This course improved my knowledge and awareness of the topic. | 0 | 0 | 0 | 15.6 | 84.4 | 100 |
| The expression and presentation of the material by the instructor (the method of instruction) were appropriate. | 0 | 0 | 0 | 20 | 80 | 100 |
| The material presented in this course was relevant to my job. | 0 | 0 | 0 | 0 | 100 | 100 |
| I was generally satisfied with the course. | 0 | 0 | 0 | 6.7 | 93.3 | 100 |
| I will recommend this course to others. | 0 | 0 | 0 | 8.9 | 91.1 | 100 |
Learning level: Table 2 presents descriptive indices, including the mean and standard deviation (SD) of the two groups on the pre- and post-test for the learning level.
The Mean and SD of Pre- and Post-test Scores of ICD-10 Coding Guidelines of Cancer
| Group | No. | Mean ± SD |
|---|---|---|
| Pre-test | 45 | 4.20 ± 2.982 |
| Post-test | 45 | 15.20 ± 1.481 |
The results of the paired-samples test-test showed a significant (P = 0.000) difference between pre-and post-test scores, suggesting that this training course enhanced the learning level of the clinical coders.
5. Discussion
With the global rise in cancer, the physical, emotional, and financial burden on individuals, families, communities, and healthcare systems is increasing (25). Accordingly, cancer control and prevention greatly mitigate the burden on healthcare systems. Medical records are a rich source of clinical patient data (26); therefore, the routine collection of data about cancer and the continuous monitoring of their quality can play a major role in setting the priorities and evidence-based policy-making for controlling this disease (27). The diagnostic information in databases as an epidemiological, prospective, and evidence-based data source is coded and used in medical research (28). Moreover, a major parameter in cancer registry programs is the accurate coding of cancer diagnoses based on international coding regulations (29).
Accordingly, coders play a key role in ensuring the quality of diagnostic codes, and their training greatly contributes to this. Today, employee training is an important tool whereby organizations can improve the quality of their services, reduce the costs of the workforce, and enhance productivity and profitability (30). Meanwhile, the evaluation of training is the most important part of training programs (31). Accordingly, the present study evaluated the effects of in-service training workshops on ICD10- cancer coding instructions in 2020. This study used levels of reaction and learning from Kirkpatrick’s model. Based on the findings, participation in this course ensured 84.4% satisfaction of the coders at the reaction level and, at the learning level, increased their knowledge and awareness of cancer coding (Tables 1 and 2). Studies by Shinohara et al. (2020) using Kirkpatrick’s model for training on taking care of stroke patients showed that the training was effective at the reaction, learning, and behavior levels (22). The study by Dorri et al. on evaluating the in-service training for nurses on cardiopulmonary resuscitation with Kirkpatrick’s model suggested that the training had a positive effect on nurses at all levels of the model (21).
The results of a study by Rafiq indicated the positive effect of training at the learning level and enhancing the skills of the participants (32). The results of a study by Sadeghi et al. also demonstrated the positive effect of training at the learning level (33). The findings of these studies were consistent with the results of the present study.
Heydari's evaluation study based on Kirkpatrick's model also suggested that holding a training workshop and using novel learning methods improved the healthcare workers’ satisfaction with the educational setting and their knowledge of new teaching and learning methods (34). In the same vein, Zafirah et al. expressed that since medical knowledge and diagnostic tools are evolving, coding instructions are also changing. Therefore, continuing training programs for coders is essential. Coders’ knowledge and skills should constantly be improved to reduce the percentage of errors. As a result, hospitals should develop internal as well as external training programs for coders (35). Accordingly, the results of the present study were reported to research center managers to plan for promoting cancer coding quality.
Training coders affects their ability to assign accurate codes. Advanced education improves documents, enables the analysis of details related to patients and, in this way, leads to coordination and better outcomes (12).
Kiongo et al. examined the role of training on the quality of instructions coding in Kenya and concluded that, following the training, coding accuracy improved from 55% to 77%, and coding completeness enhanced from 96.8% to 98.9% (36). Studies by Tola et al. also showed that educational interventions effectively improve the quality of medical documents (37). It seems that the most important factor contributing to the effect of training on promoting clinical coders’ learning is their educational needs for enhancing the quality of cancer coding based on ICD-10 and the suitability of the training materials for this need.
5.1. Conclusions
One cannot deny the role of useful, effective, and continuous training for coders because their lack of knowledge or forgetting of coding will lead to the allocation of inaccurate codes.
References
-
1.
Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2018;68(6):394-424. [PubMed ID: 30207593]. https://doi.org/10.3322/caac.21492.
-
2.
Danaei M, Haghdoost A, Momeni M. An epidemiological review of common cancers in Iran; a review article. Iran J Blood Cancer. 2019;11(3):77-84.
-
3.
World Health Organization. WHO Report on Cancer: Setting Priorities, Investing Wisely and Providing Care for All. Geneva, Switzerland: World Health Organization; 2020. Available from: https://apps.who.int/iris/bitstream/handle/10665/330745/9789240001299-eng.pdf.
-
4.
Kazem Zendehdel MP. Cancer Statistics in IR Iran in 2018. Basic Clin Cancer Res. 2018;11(1):1-4.
-
5.
Earp Siqueira ADS, Goncalves JG, Xavier Mendonca PE, Elias Merhy E, Poirot Land MG. Economic Impact Analysis of Cancer in the Health System of Brazil: Model Based in Public Database. Health Sci J. 2017;11(4). https://doi.org/10.21767/1791-809x.1000514.
-
6.
Mutai C, Wakhungu J, Mutuma J. The socio economic effect of cancer on patients’ livelihoods in Kenyan house holds. Bibechana. 2017;14:37-47. https://doi.org/10.3126/bibechana.v14i0.15412.
-
7.
World Health Organization. Module 3: Early Detection. Cancer Control: Knowledge into Action: WHO Guide for Effective Programmes. Geneva, Switzerland: World Health Organization; 2007.
-
8.
Parkin DM. The role of cancer registries in cancer control. Int J Clin Oncol. 2008;13(2):102-11. [PubMed ID: 18463952]. https://doi.org/10.1007/s10147-008-0762-6.
-
9.
World Health Organization. Cancer prevention and control in the context of an integrated approach: report by the Secretariat. Geneva, Switzerland: World Health Organization; 2016. Available from: https://apps.who.int/gb/ebwha/pdf_files/EB140/B140_31-en.pdf.
-
10.
Bray F, Znaor A, Cueva P, Korir A, Swaminathan R, Ullrich A, et al. Planning and Developing Population-Based Cancer Registration in Low- or Middle-Income Settings. Lyon (FR): International Agency for Research on Cancer; 2014.
-
11.
Sigurdardottir LG, Jonasson JG, Stefansdottir S, Jonsdottir A, Olafsdottir GH, Olafsdottir EJ, et al. Data quality at the Icelandic Cancer Registry: comparability, validity, timeliness and completeness. Acta Oncol. 2012;51(7):880-9. [PubMed ID: 22974093]. https://doi.org/10.3109/0284186X.2012.698751.
-
12.
Kiongo JG, Yitambe A, Otieno GO. Improving the Quality of Clinical Coding through the Training of Health Records and Information Officers in Selected Hospitals, Nairobi City County, Kenya. J Hosp Med Manag. 2020;6(1):1.
-
13.
Mirhashemi SH, Ramezanghorbani N, Asadi F, Rangraz MH. Auditing the accuracy of medical diagnostic coding based on international classification of diseases, tenth revision. Iran Red Crescent Med J. 2020;22(9). https://doi.org/10.32592/ircmj.2020.22.9.99.
-
14.
Tang KL, Lucyk K, Quan H. Coder perspectives on physician-related barriers to producing high-quality administrative data: a qualitative study. CMAJ Open. 2017;5(3):E617-22. [PubMed ID: 28827414]. [PubMed Central ID: PMC5621953]. https://doi.org/10.9778/cmajo.20170036.
-
15.
NCRA Education Foundation. An Introduction to the cancer registry instructor's guide. Alexandria, Virginia: NCRA Education Foundation; 2018.
-
16.
Wilson R, O’Neil ME, Ntekop E, Zhang K, Ren Y. Coding Completeness and Quality of Relative Survival-Related Variables in the National Program of Cancer Registries Cancer Surveillance System 1995–2008. J Regist Manag. 2014;41(2):65-71. [PubMed ID: 25153011]. [PubMed Central ID: PMC4369759].
-
17.
Santos S, Murphy G, Baxter K, Robinson KM. Organisational factors affecting the quality of hospital clinical coding. Health Inf Manag. 2008;37(1):25-37. [PubMed ID: 18245862]. https://doi.org/10.1177/183335830803700103.
-
18.
Hennessy DA, Quan H, Faris PD, Beck CA. Do coder characteristics influence validity of ICD-10 hospital discharge data? BMC Health Serv Res. 2010;10:99. [PubMed ID: 20409320]. [PubMed Central ID: PMC2868845]. https://doi.org/10.1186/1472-6963-10-99.
-
19.
Oashttamadea R. Effects of training on coding accuracy of obstetrical diagnosis in Naili DBS Hospital. Int J Community Med Public Health. 2019;6(11):4725-30. https://doi.org/10.18203/2394-6040.ijcmph20194631.
-
20.
NHS Digital. National Clinical Coding Training Handbook. Clinical Classifications Service; 2017-18.
-
21.
Dorri S, Akbari M, Dorri Sedeh M. Kirkpatrick evaluation model for in-service training on cardiopulmonary resuscitation. Iran J Nurs Midwifery Res. 2016;21(5):493-7. [PubMed ID: 27904633]. [PubMed Central ID: PMC5114794]. https://doi.org/10.4103/1735-9066.193396.
-
22.
Shinohara M, Nakamura T, Kunikata N. A half-day stroke workshop based on the Kirkpatrick model to improve new clinical staff behavior. J Adv Med Educ Prof. 2020;8(1):10-7. [PubMed ID: 32039268]. [PubMed Central ID: PMC6946945]. https://doi.org/10.30476/jamp.2019.74874.0.
-
23.
Cullinane M, McLachlan HL, Newton MS, Zugna SA, Forster DA. Using the Kirkpatrick Model to evaluate the Maternity and Neonatal Emergencies (MANE) programme: Background and study protocol. BMJ Open. 2020;10(1). e032873. [PubMed ID: 32014872]. [PubMed Central ID: PMC7045237]. https://doi.org/10.1136/bmjopen-2019-032873.
-
24.
Kirkpatrick DL, Kirkpatrick J. Implementing the four levels: A practical guide for effective evaluation of training programs: Easyread super large 24pt edition. Berrett-Koehler Publishers; 2009.
-
25.
Silva RCFD. [Guide to Cancer Early Diagnosis]. Rev Bras de Psiquiatr. 2019;63(1):41-2. Portuguese. https://doi.org/10.32635/2176-9745.RBC.2017v63n1.188.
-
26.
Arani LA, Hosseini A, Asadi F, Masoud SA, Nazemi E. Intelligent Computer Systems for Multiple Sclerosis Diagnosis: a Systematic Review of Reasoning Techniques and Methods. Acta Inform Med. 2018;26(4):258-64. [PubMed ID: 30692710]. [PubMed Central ID: PMC6311112]. https://doi.org/10.5455/aim.2018.26.258-264.
-
27.
Modirian M, Rahimzadeh S, Cheraghi Z, Khosravi A, Salimzadeh H, Kompani F, et al. Quality evaluation of national cancer registry system in Iran: study protocol. Arch Iran Med. 2014;17(3):193-7. [PubMed ID: 24621363].
-
28.
Altoijry A, Al-Omran M, Lindsay TF, Johnston KW, Melo M, Mamdani M. Validity of vascular trauma codes at major trauma centres. Can J Surg. 2013;56(6):405-8. [PubMed ID: 24284148]. [PubMed Central ID: PMC3859783]. https://doi.org/10.1503/cjs.013412.
-
29.
Wei KR, Wei K, Liu SC, Wei D, Liang Z, Chen W. Auto-coding of cancer registry data in China. Asian Pac J Cancer Prev. 2016;17(6):3021-3. [PubMed ID: 27356728].
-
30.
Mohammed Saad A, Mat N. Evaluation of effectiveness of training and development: the Kirkpatrick model. Asian J Bus Manag Sci. 2013;2(11):14-24.
-
31.
Topno H. Evaluation of Training and Development: An Analysis of Various Models. IOSR J Bus Manag. 2012;5(2):16-22. https://doi.org/10.9790/487x-0521622.
-
32.
Rafiq M. Training Evaluation in an Organization using Kirkpatrick Model: A Case Study of PIA. J Entrepreneu Organ Manag. 2015;4(3):152-62. https://doi.org/10.4172/2169-026x.1000151.
-
33.
Sadeghi Y, Gharlipour Z, Dashti Z, Kabiri P, Heydari S. [Survey the effectiveness of" Scientometrics" workshop for the faculty members and staff in Qom University of Medical Sciences basded on Kirkpatrick's model in 2016]. Zanko J Med Sci. 2017;18(58):68-79. Persian.
-
34.
Heydari MR, Taghva F, Amini M, Delavari S. Using Kirkpatrick's model to measure the effect of a new teaching and learning methods workshop for health care staff. BMC Res Notes. 2019;12(388):1-5. https://doi.org/10.1186/s13104-019-4421-y.
-
35.
Zafirah SA, Amrizal MN, Sharifah E, Aljunid SM. Incidence of clinical coding errors and implications on casemix reimbursement in a teaching hospital in Malaysia. Malaysian J Public Health Med. 2017;17(2):19-28.
-
36.
Kiongo JG, Otieno GO, Yitambe AO. Effects of Training on Quality of Clinical Coding at Mbagathi County Referral Hospital, Nairobi City County, Kenya. J Innov Educ Res. 2018;6(1):231-7. https://doi.org/10.31686/ijier.vol6.iss1.943.
-
37.
Tola K, Abebe H, Gebremariam Y, Jikamo B. Improving Completeness of Inpatient Medical Records in Menelik II Referral Hospital, Addis Ababa, Ethiopia. Adv Public Health. 2017;2017:1-5. https://doi.org/10.1155/2017/8389414.