1. Background
In 2017, for the first time in the world, a scale called HACOR, including 5 variables of heart rate, acidosis, consciousness, oxygenation, and respiratory rate (HACOR), was designed to predict the success and failure of non-invasive ventilation (NIV) in hypoxemia for various reasons. This score provides an easy-to-use tool for effectively identifying patients at a high risk of NIV failure. The HACOR score at 1 hour after NIV will be measured by physicians, respiratory therapists, and nurses for patients who were admitted to the ICU for the use of NIV due to hypoxemic respiratory failure. The highest score of this index is 25, and patients with a HACOR score above 5 one hour after the NIV onset have a high risk of failure (1). Having a suitable criterion to assess the success rate of NIV is important because it predicts the early detection of success or failure in NIV and intubation and can significantly reduce mortality (2). Studies in 2020 have confirmed the effectiveness of this tool and have shown that the HACOR score can be a good tool for predicting NIV failure (3). In a number of studies, this score was designed and used for respiratory failure due to various causes such as bacterial pneumonia, lung cancer, pulmonary embolism, and heart failure, in which a HACOR score < 5 one hour after the CPAP onset, with sensitivity, specificity, positive predictive value, negative predictive value, and accuracy above 70% has been presented as a suitable method to predict the NIV failure (1, 3).
Coronavirus disease 2019 (COVID-19) is a global healthcare problem that has affected the lives of many people all around the world (4); it is a type of acute respiratory pneumonia that has spread rapidly on an epidemic scale since 2019 and exacerbates the rate of mortality (5, 6). Approximately 5% of people with COVID-19 require admission to intensive care units (ICUs) (7), that most of them include people over the age of 60 with hypertension, diabetes, cardiovascular disease, and obesity (8, 9).
In this ward, sometimes NIV is used as respiratory support for patients with spontaneous respiration, stable hemodynamics, and low airway secretions who do not require emergency intubation (10). NIV is a type of respiratory support without endotracheal intubation. It is done through two methods: One by applying continuous positive airway pressure (CPAP) and the other by applying bi-level positive airway pressure (BIPAP) through a suitable nasal mask or a face mask designed for this type of ventilation. It is used in various types of respiratory failures (11-13). The benefit of NIV is the ability to talk, cough, and swallow normally. Based on recent studies, it seems that CPAP has been effective in oxygenation and gas exchange improvement and reduced the need for endotracheal intubation (1, 2). Since we cannot use deep sedation in NIV due to the need for spontaneous breathing, NIV is used only in cases of mild (partial pressure of oxygen in arterial blood/fraction of inspired oxygen (PaO2/FiO2) ratio between 200 and 300) to moderate (PaO2/FiO2 ratio between 100 and 200) hypoxia (10).
Numerous scientific studies conducted in the United States in 2019 have reported the importance of using NIV in COVID-19 patients with mild to moderate acute respiratory distress syndrome (ARDS) (2). Despite many benefits of using the HACOR score in predicting the success rate of NIV in patients admitted to the ICU, few studies have been conducted around the world on the use of the HACOR score to evaluate the success rate of NIV in hypoxemic patients with COVID-19. Based on those studies, a HACOR score < 5 one hour after the CPAP onset, with sensitivity, specificity, positive predictive value, negative predictive value, and accuracy above 70%, has been presented as a suitable method to predict the NIV failure rate in these patients, experts believe that the mortality of invasive ventilation in these patients is high and extubation is difficult to perform in many of them (14).
Applying this score in respiratory intensive care units (RICUs) at the right time to shift from non-invasive to invasive ventilation can be significantly effective in better patient management, reducing mortality and increasing the accuracy of nurses given that no study in the Iranian clinical system used the HACOR score to predict the success rate of CPAP has been so far. Therefore, the present research with the purpose evaluates the diagnostic accuracy of the HACOR score in predicting the success rate of CPAP in COVID-19 patients in Iran. We measured the cut-off point, the accuracy, the sensitivity and specificity indices, and positive and negative predictive values for one hour after CPAP in predicting its success rate of it.
2. Objectives
The present study aimed to evaluate the diagnostic accuracy of the HACOR score in predicting the success rate of CPAP in COVID-19 patients with mild to moderate ARDS hospitalized in ICUs of Shoushtar City.
3. Methods
3.1. Participants
This retrospective cross-sectional study was performed from March 21 to November 21, 2021, by convenience sampling method. A total of 200 patients with COVID-19 admitted to the RICU of Khatam Al-Anbia (PBUH) Hospital in Shoushtar City were recruited. To determine the required sample size, according to the ratio formula, the matching paper of Guia et al. (14), the proportion of people who experienced CPAP failure was P = 0.25, The error rate (α = 0.05) and accuracy (d = 0.06) were considered. Eventually, the sample size was estimated to be 200 people.
3.2. Eligibility Criteria for Participants
The participants included people over 15 years of age with PaO2/FiO2 ratio less than 300 mmHg and PaCO2 level less than 45 mmHg at room temperature or oxygen therapy with 28% FiO2. The exclusion criteria included immediate cardiac and respiratory arrest, inability to protect the airway, severe hemodynamic instability (arterial blood pressure less than 65 mmHg despite vasopressor support), severe restlessness of the patient (Richmond Agitation-Sedation Scale (RASS) > 2), and multiple organ system failures (MOSF).
3.3. Tools
The data collection tool consisted of two parts: (1) Demographic characteristics (age, sex, marital status, underlying diseases, and BMI); (2) HACOR score. It was a scale designed by Duan et al. to predict the success and failure of NIV in hypoxemia for various reasons (1). This scale has 5 variables, including heart rate, pH, Glasgow Coma Scale (GCS), PaO2/FiO2 ratio, and respiratory rate. The information needed to calculate the HACOR score one hour after CPAP, including some vital signs such as heart rate and respiratory rate, arterial blood gas analysis (for values of pH and PaO2/FiO2 ratio), and level of consciousness (using GCS) were obtained for each patient, a score was assigned to each variable based on the tool. Then, the total score of 5 variables is calculated. The highest score of this scale is 25 and patients with a HACOR score above 5 one hour after the CPAP onset have a high risk of failure (1).
3.4. Procedure
To collect data, the files of COVID-19 patients who used CPAP mode were examined. CPAP was used in these patients using a proper-sized oronasal mask. CPAP settings were started at 8 cmH2O pressure and titrated with FiO2 according to patient comfort to improve oxygenation and respiratory pattern and maintain SpO2 > 94%. The prone position was applied from the beginning of admission in the RICU 2 - 3 times a day, each time for 2 - 3 hours, during the entire hospital stay according to patient tolerance. Proning was only applied after the HACOR evaluation.
Data were collected by demographic checklist and the HACOR score. Patients were then analyzed based on the results of CPAP mode (success or failure). Failure was defined as the need for intubation or death. Criteria for endotracheal intubation included hemodynamic instability (arterial blood pressure less than 65 mmHg despite the use of vasopressor), decreased level of consciousness (GCS less than 9), respiratory rate above 40 bpm, respiratory fatigue symptoms, PaO2/FiO2 ratio below 150 mmHg for more than 48 hours using the CPAP mode, agitation (RASS score > 2) and objective criteria and clinical decision for intubation were taken into account (14).
3.5. Ethical Considerations
After the proposal was approved by the Research Committee and the code of ethics was obtained from the Ethics Committee of the Shoushtar University of Medical Sciences (Code: IR.SHOUSHTAR.REC.1400.017), the necessary permits to collect information were issued to Khatam Al-Anbia (PBUH) Hospital by the university. Confidentiality of all information received from the files was emphasized.
3.6. Statistical Analysis
Normality distribution of the data was done using the Kolmogorov-Smirnov test. t-test was used to compare the means, and the paired t-test was used to compare the variables before and one hour after CPAP. The receiver operating characteristics (ROC) curve, sensitivity and specificity indices, and positive and negative predictive values were used to determine the diagnostic value of the method. The significance level was set at P ≤ 0.05, and the data were analyzed by SPSS software (version 22).
4. Results
A total of 200 COVID-19 patients admitted to the ICU from March 21 to November 21, 2021, in Khatam Al-Anbia (PBUH) Hospital in Shoushtar were recruited. The mean ± SD age of patients was 63.23 ± 16.23. Thirty-eight percent (n = 76) of them were female, and 95% (n = 190) were married.
Further, 27.5% (n = 55) had no underlying diseases, 42.5% (n = 85) had diabetes, and 35% (n = 70) had high blood pressure. The CPAP method was successful in 21.5% (n = 43) and failed in 78.5% (n = 157) of patients.
Comparison of the HACOR score and its subscales before and one hour after the CPAP method is highlighted in (Table 1). The HACOR score and its subscales differed significantly before and one hour after the CPAP onset (P < 0.001. In other words, one hour after the CPAP onset, the HACOR score was lower, followed by an improvement in heart rate, level of consciousness, oxygenation, and respiration rate.
Table 1.Comparison of Heart Rate, Acidosis, Consciousness, Oxygenation, and Respiratory Rate; Heart Rate; pH; Glasgow Coma Scale; Partial Pressure of Oxygen in Arterial Blood/Fraction of Inspired Oxygen; and Respiratory Rate Before and One Hour After the Continuous Positive Airway Pressure Method a
Abbreviations: CPAP, continuous positive airway pressure; HACOR, heart rate, acidosis, consciousness, oxygenation, and respiratory rate; HR, heart rate; hbpm, heart beats per minute; pH, hydrogen ion concentration; GCS, Glasgow Coma Scale; PaO2/FiO2, partial pressure of oxygen in arterial blood/fraction of inspired oxygen; RR, respiratory rate; bpm, breaths per minute.
a Values in the table are presented as mean ± SD, and a paired t-test was used for data analysis.
b P < 0.01
The ROC curve and the area under the curve (AUC) were used to evaluate the efficiency of the HACOR score and the PaO2/FiO2 ratio in predicting the success or failure of CPAP in COVID-19 patients.
The AUC for the HACOR score in the diagnosis of success or failure of the CPAP of COVID-19 was 0.776 (CI: 0.709 - 0.844, P < 0.001) before the CPAP and 0.968 (CI: 0.941 - 0.995, P < 0.001) one hour after the CPAP. The optimal critical point for the HACOR score was 8.5 before CPAP (sensitivity of 0.632 and specificity of 0.857) and 5 after the CPAP (sensitivity of 98.06% and specificity of 83.33%). This success may be related to higher pH, GCS, PaO2/FiO2 ratios, and lower heart rate (HR) and respiratory rate (RR) rates one hour after the CPAP in successful individuals than in those who failed (Figure 1).
Figure 1.
Receiver operating characteristic (ROC) curve of the heart rate, acidosis, consciousness, oxygenation, and respiratory rate (HACOR) score before and one hour after the continuous positive airway pressure (CPAP) method to predict the CPAP success rate
The cut-off point for the HACOR score one hour after CPAP was 5, which had a sensitivity of 98.06% (94.45% - 99.60%) and a specificity of 83.33% (68.64% - 93.03%). In this sample, 35 (92.1%) of patients with a HACOR score less than 5 were improved, and 3 (7.9%) had CPAP failure, which resulted in a negative predictive value of 92.11% (79.05% - 97.30%). Moreover, 152 (95.6%) patients with a HACOR score greater than 5 experienced CPAP failure, and 7 (4.4%) were improved, with a positive predictive value of 95.60% (91.69% - 97.71%). In addition, the accuracy of the HACOR score one hour after CPAP was 94.92% (90.86% - 97.54%), indicating that this method can predict the success and failure of CPAP in COVID-19 patients admitted to the ICU.
Comparison of demographic variables, HACOR score, and its subscales in the success and failure rate of the CPAP method are highlighted in (Table 2). The age variable in people who experienced CPAP failure was 6.99 ± 2.75 more than those who had a successful CPAP result (P = 0.020). Gender and marital status were also not effective in the success or failure of CPAP (P = 0.378). The BMI variable in people who experienced CPAP failure was 1.68 ± 0.75 more than those whose CPAP was successful (P = 0.018). Underlying diseases (P = 0.442), HTN (P = 0.459), and heart disease (P = 0.361) have not been effective in the success or failure of CPAP, but diabetes has been effective. Diabetes disease in people who experienced CPAP failure was 43% more than those whose CPAP was successful (P = 0.039).
Table 2.Comparison of Demographic Variables, Heart Rate, Acidosis, Consciousness, Oxygenation, and Respiratory Rate Score, and Its Subscales in the Success and Failure Rate of the Continuous Positive Airway Pressure Method a
| Variables | Success CPAP (HACOR One Hour After CPAP < 5; n = 43) | Failure CPAP (HACOR One Hour After CPAP > 5; n = 157) | P-Value |
|---|---|---|---|
| Age | 58.48 ± 15.86 | 65.48 ± 16.06 | 0.02 b |
| Gender | 0.378 | ||
| Female | 19 (44.2) | 57 (36.3) | |
| Male | 24 (55.8) | 100 (63.7) | |
| Marital status | 0.693 | ||
| Married | 42 (97.7) | 148 (94.3) | |
| Single | 1 (2.3) | 9 (5.7) | |
| Underlying disease | 0.442 | ||
| Yes | 29 (67.4) | 116 (73.9) | |
| No | 14 (32.6) | 41 (26.1) | |
| DM | 0.039 c | ||
| Yes | 24 (55.8) | 60 (38.2) | |
| No | 19 (44.2) | 97 (67.8) | |
| Hypertension | 0.459 | ||
| Yes | 13 (30.2) | 57 (36.3) | |
| No | 30 (69.8) | 100 (63.7) | |
| Cardiovascular disease | 0.361 | ||
| Yes | 2 (4.7) | 14 (8.9) | |
| No | 41 (95.3) | 143 (91.1) | |
| BMI | 29.02 ± 3.58 | 30.70 ± 4.46 | 0.018 b |
| HACOR.pre | 7.04 ± 1.86 | 9.70 ± 3.06 | 0.000 d |
| HACOR.post | 3.50 ± 1.57 | 9.20 ± 3.04 | 0.000 d |
| pH.pre | 7.38 ± 0.05 | 7.34 ± 0.07 | 0.000 d |
| pH.post | 7.39 ± 0.04 | 7.33 ± 0.08 | 0.000 d |
| RR.pre | 30.95 ± 5.58 | 33.66 ± 5.91 | 0.008 d |
| RR.post | 26.30 ± 2.50 | 31.25 ± 5.80 | 0.007 d |
| HR (hbpm).pre | 111.25 ± 18.55 | 116.84 ± 16.84 | 0.004 b |
| HR (hbpm).post | 106.27 ± 16.40 | 114.33 ± 17.01 | 0.006 d |
| PaO2/FiO2.pre | 96.87 ± 7.14 | 89.13 ± 16.67 | 0.006 d |
| PaO2/FiO2.post | 153.84 ± 27.42 | 103.55 ± 16.43 | 0.004 d |
| GCS.pre | 15.0 ± 0.00 | 14.54 ± 1.14 | 0.01 d |
| GCS.post | 15.0 ± 0.00 | 14.24 ± 1.60 | 0.002 d |
Abbreviations: CPAP, continuous positive airway pressure; HACOR, heart rate, acidosis, consciousness, oxygenation, and respiratory rate; pH, hydrogen ion concentration; RR, respiratory rate; HR, heart rate; hbpm, heart beats per minute; PaO2/FiO2, partial pressure of oxygen in arterial blood/fraction of inspired oxygen; GCS, Glasgow Coma Scale.
a Values in the table are presented as mean ± SD or No. (%) and a t-test was used for data analysis.
b P < 0.05
c P < 0.1
d P < 0.01
The PaO2/FiO2 ratio was significantly lower in patients with CPAP failure. On the other hand, one hour after the CPAP onset, significant improvements were observed in HR rates, pH, PaO2/FiO2 ratio, and RR rates.
The AUC for the PaO2/FiO2 ratio in the diagnosis of success or failure of the CPAP of COVID-19 was 0.732 (CI: 0.651 - 0.813, P = 0.041) before CPAP and 0.985 (CI: 0.970 - 1.0, P = 0.008) one hour after CPAP. The optimal critical point for the PaO2/FiO2 ratio was 91.55 before CPAP (sensitivity of 0.714 and specificity of 0.684) and 131 after CPAP (Figure 2).
Figure 2.
Receiver operating characteristic (ROC) curve of the partial pressure of oxygen in arterial blood/fraction of inspired oxygen (PaO2/FiO2) ratio before and one hour after the continuous positive airway pressure (CPAP) method to predict the CPAP failure rate
The cut-off point for the PaO2/FiO2 ratio one hour after the CPAP was 131, with a sensitivity of 85.71% (71.46% - 94.57%) and a specificity of 98.72% (95.45% - 99.84%). Further, 36 (94.7%) patients with a PaO2/FiO2 ratio higher than 131 were improved, and 2 (5.3%) had CPAP failure, which led to a negative predictive value of 96.25% (92.44% - 98.18%). Moreover, 154 (96.3%) patients whose PaO2/FiO2 ratio was less than 131 experienced CPAP failure, and 6 (3.8%) patients were improved, with a positive predictive value of 94.74% (81.87% - 98.63%). In addition, the accuracy of the PaO2/FiO2 ratio one hour after CPAP was 95.96% (92.19% - 98.24%). The sensitivity, specificity, and accuracy of the PaO2/FiO2 ratio one hour after CPAP indicate that this method is capable of predicting the success and failure of CPAP in COVID-19 patients admitted to the ICU.
5. Discussion
This study investigated the diagnostic accuracy of the HACOR score in predicting the success rate of CPAP in COVID-19-related hypoxemia. The HACOR score can significantly reduce nosocomial mortality to detect the early failure or success of NIV and to predict the appropriate intubation time (14-16).
In the present study, ROC showed that the HACOR score one hour after CPAP could predict the success and failure of CPAP in patients with COVID-19-induced hypoxemia admitted to the ICU. This result is in line with the results reported by Guia et al. on COVID-19 patients with acute respiratory failure (ARF) to determine the benefits of using the HACOR score in predicting the success rate of CPAP in COVID-19-related hypoxemia, Innocenti et al. on patients with ARF to predict mortality in NIV and Al-Rajhi et al. on patients with community-acquired pneumonia to evaluate the results and predict NIV failure (14-16).
The results of the present study suggest that the cut-off point for the HACOR score one hour after CPAP, with sensitivity, specificity, positive predictive value, negative predictive value, and accuracy above 70%, was equal to 5. In various studies on hypoxemic patients with respiratory failure, a HACOR score < 5 one hour after the CPAP onset, with sensitivity, specificity, positive predictive value, negative predictive value, and accuracy above 70%, has been presented as a suitable method to predict the NIV failure rate and reduce in-hospital mortality (1, 3, 14, 16).
As for the use of this score to determine various causes of hypoxemic respiratory failure, in the study of Duan et al. in 2017, the HACOR score was designed for respiratory failure due to various causes such as bacterial pneumonia, lung cancer, pulmonary embolism, and heart failure, whose mechanism of action is not always the same as that of acute respiratory failure (pneumonia) caused by SARS-CoV-2 (1). COVID-19 pneumonia has specific features, including a combination of damages caused by direct viral cytopathic effects and indirect cytokine storms (17). This study examined this indicator specifically in COVID-19 patients.
The results also suggest that the sensitivity, specificity, and accuracy of the PaO2/FiO2 ratio one hour after the CPAP showed that this method is also may be approximately as large as the HACOR score itself, a determining factor and even an easier indicator in predicting the CPAP failure rate, which is in line with the results of Guia et al. (14). Also, the cut-off point for PaO2/FiO2 ratio one hour after CPAP was 131. The cut-off point for the PaO2/FiO2 ratio score one hour after the CPAP in Guia et al.’s study was 152 mmHg (14), indicating that patients with more severe hypoxemia than a similar study experienced CPAP failure and high mortality.
Studies have indicated that endotracheal intubation is highly required due to the rapid deterioration of COVID-19 patients with acute hypoxemic respiratory failure and the lack of significant improvement with CPAP. It is important to note that early assessment of gas exchange changes is necessary to prevent delays in endotracheal intubation (14, 18). CPAP allows for increased functional residual capacity and improves ventilation/perfusion adaptation (18). The lack of reported increase in PaO2/FiO2 ratio one hour after CPAP indicates CPAP failure, so a more invasive form of respiratory support may be required.
5.1. Limitations
Naturally, this study was not without its limitations. Including only the CPAP mode was used, not BIPAP. Another important limitation of this study is that CPAP was used only through a ventilator and an oronasal mask, which may be less effective than high-flow systems using helmet as an interface. It is suggested in future studies comparing the effectiveness of NIV, BIPAP, CPAP, helmet-CPAP and HFNC as well as predictive factors of failure of each respiratory support technique, to be able to establish more adequate success/failure predictive scales.
5.2. Conclusions
Our findings support that although the HACOR score had a good diagnostic performance in predicting the success rate of CPAP in patients with COVID-19 induced hypoxemia. PaO2/FiO2 ratio was also shown to be a good predictor of success.
