1. Context
Health care organizations use information technology for improving patient care (1). Computerized provider order entry (CPOE) combined with a clinical decision support system (CDSS) is a complex component of the electronic medical record. CPOE is an electronic application the physician or care provider uses to enter order medication, laboratory tests, request imaging, blood bank, and request a consultation. The primary purposes of CPOE include assurance of legibility and completeness of provider orders (2, 3). Much of the potential value of CPOE come from clinical decision support tools that can help a physician improve clinical decision-making at the point of care (4).
CDSS in provider order entry system includes drug-drug interaction checking, drug-allergy checking, drug-age checking, drug dose calculating, restricting transfusion orders, and detecting unnecessary diagnostic tests (5). The CDSS is embedded in medication orders, transfusion orders, requests for medical images, and laboratory test order. CDSS embedded in CPOE contributes to improving patient safety, preventing waste of the cost of care, and promoting health care services (4).
Evaluation of CDSS is required to ensure long-term CDSS integrated into CPOE sustainability, improve care quality (process, structures, and patient outcome). Also, evaluation is essential to ensure the appropriateness of the CDSS. Feedback obtained from the assessment of CDSS performance help to improve the suitability of CDSS.
Despite the increasing use of CPOE combined with CDSS for improving patient safety and reducing the use of services, the effects of CPOE combined with CDSS are unclear. Several studies have examined the impact of clinical decisions integrated into the CPOE system (6-12).
2. Objectives
One study specifically explored the effects of CDSS (13); however, many questions have remained unanswered about CPOE combined with CDSS impact. Then, having the criteria for evaluating the impact of CPOE combined with CDSS is necessary. Accordingly, the present study was conducted as a scoping review to identify the criteria for assessing the effects of CDSS integrated with CPOE.
3. Methods
This scoping review investigated the effects of CDSS integrated into CPOE. Any types of clinical decision support (alert, reminder, info button, and relevant information) presented to physicians during medical order entry (from initial order to submit the order) and published until February 2019 in English were included in the present study. However, thesis/dissertation, proceeding papers, conference papers, letters to the editor, short communications, unpublished articles, and non-English language papers were excluded in the final stage of screening.
To collect relevant studies, a systematic search was carried out using databases from inception to February 2019. To make sure that there were no other similar systematic reviews, a rapid search was conducted on Cochrane library databases to find a possible systematic article related to the metrics evaluating the effects of intervention (medical order entry system with decision support). Then, nine information databases, including PubMed, Embase, ProQuest, Scopus, Web of Science, Cochrane, Science Direct, ACM digital library, IEEE Xplore Digital Library, were searched. In all search stages, the results were reviewed by other individuals and approved.
The search was conducted with relatively common terms using synonymous words and "OR" operators. To achieve more specificity, the search was implemented with synonymous words and the "AND" operator. The "MeSH terminology" keyword in the PubMed database and the "Emtree terminology" keyword in the Embase database were also used to find terms related to articles (Table 1). The search strategy was approved by a senior medical librarian.
| Database | PubMed, Embase, ProQuest, Scopus, Web of Science, Cochrane, Science Direct, ACM digital library, IEEE Xplore Digital Library |
|---|---|
| Search strategy | #1 AND #2 AND #3 |
| #1 | "Medical Order Entry Systems" OR "CPOE" OR "Computerized Order Entry" OR "Computerized Prescriber Order Entry" OR "Computerized Provider Order Entry" OR "Electronic Order" OR "Electronic Prescribing" OR "Electronic Physician Order Entry" OR "Computerized Physician Order Entry" |
| #2 | "Clinical Decision Support Systems" OR "Clinical Computerized Decision Support Systems" OR "Decision-Support Systems" OR "Reminder Systems" OR "Computer-Assisted Decision-Making" OR " Computer-Assisted Therapy " OR "Expert Systems" OR "Alert System" OR "Clinical Decision Support Alert" OR "Formulary Decision Support" |
| #3 | "Impact" OR "Effect" OR "Mortality" OR "Length Of Stay" OR "Medication Errors" OR "Drug Side Effects" OR "Adverse Reactions" OR "Readmission" OR "Hospitalization Cost" OR "Treatment Cost" OR "Hospital Acquired Infection" OR "Nosocomial Infection" OR "Patient Outcome" OR "Clinician Performance" OR "Satisfaction" |
Search Strategy to Investigate the Effects of Computerized Provider Order Entry Combined with Decision Support Systems
The required information was extracted using a collection form. This form included the journal name, article title, study design, study date, research sample, data collection tools, intervention (medical order entry with decision support), intervention effects, and conclusion. All forms were developed item by item. In the extraction stage, one of the authors inserted the data into the form, and other authors reassurance them. In case of disagreements between the two authors, a third author resolved the issue.
Two authors screened the titles and abstracts. Data abstracted were categorized according to the medical order type (medication order, radiology orders, laboratory order, and blood bank order). The effects of CPOE combined with CDSS were captured using a narrative review. Effects of electronic medication orders with CDSS were classified using the taxonomy described by Bright et al. (13).
4. Results
The literature search was conducted in February 2019 initially and identified a total of 8963 possible articles, and after removing duplicates, 7162 records remained. A three-stage review process was employed (title, abstract, and full text). In the first and second stages, two reviewers screened all titles and abstracts for relevancy and eliminated articles that did not meet the inclusion criteria. Following the first-stage screening, 301 articles were retrieved and reviewed, and one reviewer conducted the third-stage screening process. Further 125 eligible studies were included. The flow diagram of study selection is shown in Figure 1. Then, 38 studies were eliminated because their publication date was before 2013, and there were similar studies published after 2013. However, the publication date of four studies was before 2013, and there was not similar study after 2013. Finally, 87 studies were included.
PRISMA flow diagram
The effect of decision support combined with the CPOE was classified by the order type. The largest effects item of decision support with the medical order entry system belonged to medication orders. Most of the studies had evaluated the effectiveness of decision support during medication order on medication errors, length of stay in the ward and hospital, mortality in the ward and hospital, compliance with standardizing therapy, and physician response to alerts. However, very few publications had evaluated the impact of decision support during a medication order on re-admission, mechanical ventilation duration, antimicrobial resistance, user workload, and efficiency time. Twenty-six effects related to the decision support integrated into the medication order entry system are reported in Table 2.
| Number | Categoriy | Reference Count | Reference |
|---|---|---|---|
| Clinical effects | |||
| 1 | Medication errors | 17 | (6, 9-11, 14-28) |
| 2 | Hospital mortality | 5 | (11, 17, 25, 29-31) |
| 3 | Length of stay | 7 | (11, 17, 19, 25, 30, 32, 33) |
| 4 | Adverse drug event | 3 | (25, 29, 34) |
| 5 | Re-admission | 1 | (17) |
| 6 | Mechanical ventilation duration | 1 | (35) |
| 7 | Patient safety/falling rate | 4 | (21, 22, 36-38) |
| 8 | Antimicrobial resistance | 1 | (30) |
| Process of care effects | |||
| 9 | Adherence to standardized care | 13 | (16, 19, 23, 30, 36, 39-49) |
| 10 | Medication discontinuation | 3 | (21, 31, 50) |
| 11 | The time of the chemotherapy process | 1 | (6) |
| User workload and efficiency effects | |||
| 12 | Clinician and clinical pharmacy Workload | 4 | (14, 23, 36, 51) |
| 13 | Time required to enter medication order | 2 | (19, 52) |
| 14 | Workflow (changes in work practices) | 1 | (19) |
| 15 | Staff time efficiency | 1 | (19) |
| 16 | Reduction in team-wide discussions | 1 | (19) |
| 17 | Education | 1 | (53) |
| 18 | Pharmacist–physician communication | 1 | (54) |
| 19 | Alert burden on users | 2 | (19, 55) |
| Economic effects | |||
| 20 | Drug costs | 5 | (17, 29, 30, 56, 57) |
| 21 | Usage of medication | 3 | (19, 30, 58) |
| Use and implementation effects | |||
| 22 | Frequency of alerts | 7 | (19, 32, 50, 55, 59-62) |
| 23 | Physician response to an alert | 3 | (31, 59, 62) |
| 24 | Number of orders canceled | 1 | (63) |
| 25 | Usability | 2 | (9, 19) |
| 26 | Rate of order set use | 1 | (64) |
Effects of Decision Support Integrated into Medication Order Entry System
The majority of studies had evacuated the effectiveness of decision support during a medical image order entry on medical image appropriate order and medical image utilization. On the other hand, fewer studies had evaluated the impact of decision support during image requests on turnaround time, repeated tests, and user compliance to the regulations. Six effects related to decision support during medical image requests are shown in Table 3.
Effects of Decision Support During Medical Image Request
Most studies had explored the effectiveness of decision support integrated into laboratory test order entry on usage of laboratory resources, unnecessary and duplicate orders, costs, and user satisfaction. However, only a few studies had evaluated the impact of decision support into laboratory test order entry on workflow, user compliance with guidelines, mortality, and length of stay. Eleven effects related to decision support integrated into laboratory test order entry are outlined in Table 4.
| Number | Effects | Reference Count | Reference |
|---|---|---|---|
| 1 | Usage of laboratory resource | 5 | (77-81) |
| 2 | Mortality (secondary outcome) | 1 | (80) |
| 3 | Length of stay | 1 | (80) |
| 4 | Cost | 4 | (77, 80, 82, 83) |
| 5 | User satisfaction | 2 | (79, 83) |
| 6 | Compliance with guideline | 1 | (12) |
| 7 | Unnecessary and duplicate ordering | 3 | (82, 84, 85) |
| 9 | Effective patient care | 1 | (86) |
| 10 | Change physician test-ordering behavior | 1 | (83) |
| 11 | Impact on workflow | 1 | (83) |
Effects of Decision Support Integrated into Laboratory Test Order Entry
Many studies had assessed the effectiveness of decision support integrated into blood products orders entry on the usage of blood products, compliance with guidelines, and reducing the use of unnecessary blood products. However, sparse research had tested the impact of decision support integrated into blood products order entry on blood product costs, mortality, and length of stay. Six effects related to decision support integrated into blood products order entry are summarized in Table 5.
Effects of Decision Support Integrated into Blood Products Order Entry
5. Discussion
This study showed that the main category evaluation criteria for CDSS integrated into CPOE included four categories of evaluation criteria for decision support integrated into medication order entry system, decision support during medical image order entry, decision support integrated into laboratory test order entry, and decision support integrated into blood products order entry. The main evaluation criteria can be regarded as the process of care, structure, and patient outcome. To the best of our knowledge, this is the first scoping review determining the criteria for evaluating the effects of implementation CDSS integrated into the CPOE in the clinical setting. These types of studies are carried out for preliminary evaluation of the scope of available research literature on the effects of CPOE combined with CDSS.
CPOE combined with CDSS can help physicians in decision-making at the point of care. CDSS integrated into CPOE is employed to prevent medication error, improve patient care, enhance patient safety, reduce costs, and increase physician adherence to standardized care. Decision support can be embedded into all types of medical order, including medication order, laboratory test order, medical image order, and blood product requests. According to this review, most studies had dealt with the decision support embedded within the medication order entry system on medication error (drug-drug interaction, drug-allergy interaction, drug-lab interaction, dose error, and drug-condition interaction). Clinical decision support embedded within medication order reduces medication error. Ranji et al. found that CDSS integrated into CPOE dramatically reduced medication error (18). The primary objective of CPOE combined with CDSS is to reduce medication error. Then, CPOE with CDSS prevents medication error. On the other hand, some studies have found that CPOE facilitated medication error (97).
According to this study results, the effects of the implementation of clinical decision support integrated into medication order entry system should be examined on adverse drug events since it is unclear (98) and there is not a uniform way to collect adverse drug event data.
Some studies had evaluated the effects of implementing the clinical decision support embedded within medication order on hospital mortality and length of stay in a clinical setting. Prgomet et al. found that computerized decision support embedded within CPOE reduced hospital mortality rate and length of stay (11). Most studies assessing the effect of CPOE with CDSS on hospital mortality and length of stay did not apply a high-quality method, such as randomized control trial.
Many studies had evaluated the effects of implementation of CDSS integrated into a CPOE on adherence to standardize care. Many studies had demonstrated that E-prescribing with decision support could enhance adherence to standardize care (44, 46). One of the best strategies to conduct health care policy is implementing policy on CPOE.
Decision support embedded within medication orders affects the rate of medication discontinuation. On the other hand, few studies had evaluated the effects of decision support integrated into CPOE on this rate (21). Close loop workflow integrated with CDS embedded within CPOE can reduce the rate of medication discontinuation and help in tracking orders.
The effects of implementation of decision support embedded within medication order on clinician and clinical pharmacy workload are controversial. Sparse studies have been carried out on the effect of medication order with decision support (36, 51). Clinical decision support embedded within CPOE can increase workload because of the need to taking more steps for the entry of information.
Regarding the impact of decision support embedded within medication order on drug cost and drug usage, Fischer et al. found that CPOE with a CDSS would reduce the drug cost and drug usage (99) because drug cost affects clinician decision-making regarding care and clinical decision support can restrict inappropriate prescription.
Many studies, such as a study conducted by McCoy et al. (100) had evaluated physician response to decision support embedded within the medication order entry system. Nevertheless, a few studies have been conducted on the usability and functionality of clinical decision support embedded within the CPOE. Usability evaluation is imperative for assurance of learnability, ease of use, memorizing, and user satisfaction with the information system.
The majority of studies had evaluated the effects of decision support integrated into medical image order entry system on inappropriateness order. These results are similar to those reported by Goldzweig et al. (70). Unnecessary medical image order is common. Furthermore, unnecessary medical image order increases health care service costs. As a result, the rate of the unsatisfied user from healthcare centers increases. CDSS integrated into CPOE can decrease inappropriate medical image orders.
According to our findings, most studies had been performed on decision support systems in the laboratory order context, and evaluated the effect of decision support integrated into laboratory test order entry system on laboratory resource usage and laboratory test cost. Eaton et al. observed that laboratory test order entry with decision support can reduce laboratory resource usage and laboratory test cost (78). Approximately half of the orders in health care centers are unnecessary. Clinical decision support integrated into CPOE may restrict laboratory tests, identify redundant order, and display past laboratory results.
Most studies had evaluated the impact of decision support integrated into the blood products order entry system on blood products usage and provider compliance with the guideline. Hibbs et al. found that decision support applied to transfusion order enhanced transfusion practice (89). The rule-based decision supports transfusion order and limits blood bank order, and consequently, clinician practice compliances with standard care.
This study introduced a list of criteria for evaluating the implementation of CDSS combined with CPOE. It helps identify weaknesses and strengths of CDSS combined with CPOE.
There is little evidence on the effect of CDSS combined with CPOE on user workload and efficiency. Further studies for evaluating the effects of decision support integrated into CPOE are essential. Gray literature and conference papers were excluded, which was a limitation. Also, only papers published in the English language were included.
There is evidence that there are logical scientific implications for evaluating successful implementation and effects of CDSS integrated into CPOE in the clinical setting. The findings of the present study offered the key metrics for evaluating the effectiveness of implemented decision support embedded within each medical order type. These studies provide extensive criteria to evaluate CDSS integrated into CPOE. These evaluation criteria can be used to evaluate CDSS integrated into CDSS in practice. Future studies on business intelligence development to present the effects of implementing CDSS integrated into CPOE will help policymakers to assure successful implementation of CDSS integrated into the CPOE. Representing the effects of CDSS integrated into CPOE using visual tools is effective for the management of decision-making.
