As previously stated, the aim of this research is to develop a predictive model using ML algorithms to ascertain whether a patient will experience hypotension in the subsequent 10 minutes based on a five-minute data input taken earlier. The data monitoring interval, signifying the five-minute window used for model monitoring, holds paramount importance in this study. In Experiment 4, we explored the impact of altering the data monitoring interval while keeping other parameters constant on the model's performance. We considered three distinct data monitoring intervals: Three, 10, and 15 minutes, in contrast to the original 5-minute interval.
The obtained results indicate that reducing the data monitoring interval while maintaining a consistent prediction interval significantly enhances the model's performance. In other words, the improvement in model performance resulting from the shortened data monitoring interval suggests that, under these conditions, the model places a greater emphasis on the available patient-specific information, particularly dynamic features that evolve over time. This, in turn, facilitates more effective learning by the model, leading to improved overall efficiency and enhanced prediction of hypotension.
After exploring the impact of altering the data monitoring interval, the next experiment (experiment 5) involves investigating the effect of the prediction interval on the model's predictive capabilities. In this study, the prediction interval is initially set at 10 minutes. However, the mentioned experiment aims to evaluate the model's performance using prediction intervals of 3, 5, and 15 minutes instead of the original 10-minute interval. The central hypothesis in this section posits that, while maintaining a fixed data monitoring interval (which is five minutes in this study), shorter prediction intervals will enhance the accuracy of the model's predictions. This hypothesis is based on the notion that reducing the time gap between these two intervals is likely to result in more accurate predictions by minimizing information gaps. Longer intervals, on the other hand, might lead to the exclusion of critical data, ultimately weakening the model's predictive capability. The results validate the aforementioned hypothesis, demonstrating that shorter prediction intervals indeed contribute to improved model performance in predicting hypotension.
The process of labeling the collected samples, on which the model's learning and evaluation are based, is one of the most important parts of this research. This task involves considering the concept of hypotension and the threshold associated with it. As mentioned, hypotension can be defined in two ways: Absolute or relative. In this study, the determination of the hypotension threshold was done in an absolute manner, using three dynamic variables: The MAP, DBP, and SBP. While the relative concept of hypotension can also be utilized to establish this threshold, we conducted an experiment by employing relative thresholds instead of absolute ones for data labeling. In this case, by considering the values of the three mentioned dynamic variables at time zero, if more than 20% of this value decreases in any of the subsequent minutes and in any of these three variables, we labeled the respective sample as "1", indicating the occurrence of a hypotensive event.
After training and evaluating the models with relative labeled data, it was observed that compared to the absolute labeling results, the RF model exhibited the best performance. Through relative labeling, the RF model achieved an accuracy of 82.1%. However, employing the absolute labeling approach resulted in an accuracy value of 82.7%. Therefore, no significant improvement was observed by adopting the relative labeling approach. Furthermore, it's worth noting that the study by Putowski et al. (
18) also highlighted the superiority of defining absolute hypotension over the relative definition. Given the confidence in the correctness of the obtained results and the demonstrated superiority of the absolute labeling method in this study, we can confidently assert that absolute labeling outperforms the relative labeling approach.
In the present study, as previously mentioned, the features can be categorized into two groups: Dynamic and static. Clinical interventions, such as position and type of anesthetic drug injection, are considered static features. To evaluate the impact of information obtained from these two clinical interventions in the prediction of ML models, we proceeded with two main analyses.
First, the ML models were trained solely using dynamic features. The best-performing model, RF, achieved an accuracy of 80.3%. Subsequently, we added the two mentioned clinical interventions to the dynamic feature set and compared the outcomes. The best-performing model, RF, reported an accuracy of 81.2%. The results clearly demonstrate that the inclusion of data pertaining to position and type of anesthetic drug injection as clinical interventions has a positive effect on the model's performance.
Second, we trained ML models using all static and dynamic features. The best-performing model, RF, reached an accuracy of 88.3%. Next, we removed position and type of anesthetic drug injection from the feature set and re-evaluated the models. The best-performing model, RF, reported an accuracy of 82%. Once again, the results indicate that the omission of these clinical interventions leads to a decline in performance.
To conclude, our study employed various feature selection techniques, including sequential feature selection and dimensionality reduction. The most notable enhancement in performance was observed when applying the SVD method, which is a dimensionality reduction technique. However, it is imperative to note that in methods like SVD, the selected features remain a black box and lack clarity. Given the paramount importance of model interpretability and comprehending the influential input variables to achieve desirable outcomes, we recommend exploring alternative feature selection methods that provide these attributes.