https://doi.org/10.5812/jamm-140888

Comparison of Long Short-Term Memory and Convolutional Neural Network Models for Emergency Department Patients’ Arrival Daily Forecasting

authors:

avatar Sina Moosavi Kashani ORCID 1 , avatar Sanaz Zargar Balaye Jame ORCID 2 , avatar Nader Markazi-Moghaddam ORCID 2 , * , avatar Ali Omrani Nava ORCID 3

Faculty of Industrial and Systems, Tarbiat Modares University, Tehran, Iran
Department of Health Management and Economics, School of Medicine, AJA University of Medical Sciences, Tehran, Iran
Surgery and Trauma Research Center, AJA University of Medical Sciences, Tehran, Iran
Journal of Archives in Military Medicine: Vol.12, issue 1; e140888
published online: March 2, 2024
article type: Research Article
received: September 11, 2023
accepted: February 7, 2024

How To Cite Moosavi Kashani S, Zargar Balaye Jame S, Markazi-Moghaddam N, Omrani Nava A. Comparison of Long Short-Term Memory and Convolutional Neural Network Models for Emergency Department Patients’ Arrival Daily Forecasting. J Arch Mil Med. 2024;12(1):e140888. https://doi.org/10.5812/jamm-140888.

Abstract

Background:

One of the most critical challenges in the emergency department (ED) is overcrowding, which creates negative consequences for patients and staff. Therefore, predicting the rate of patients entering the ED can help manage resources in this department effectively.

Objectives:

According to the time of data collection, we intended to predict the volume of patient admissions to the ED in epidemic conditions, such as COVID-19 and non-epidemic. In addition, we planned to compare the performance of the LSTM and CNN models.

Methods:

The collected data consists of three main categories. The first category pertains to air pollutants, provided by the Tehran air quality control organization. The second type relates to data from the Iran Meteorological Organization, and the third category includes the number of patients admitted to the ED of a hospital in Tehran. We also incorporated binary indicators for epidemic and non-epidemic conditions in the dataset. The data collection period spans from February 2018 to March 2022. We employed the Dickey-Fuller test to assess the stationarity of the data. After preprocessing, we independently developed long short-term memory (LSTM) neural network and convolutional neural network (CNN) models, considering various time windows of previous days. Keras and Tensorflow libraries in Python, along with the Google Colab environment, were utilized to execute the models.

Results:

The LSTM model exhibited the lowest root mean square error (RMSE) and mean absolute error (MAE) with a time window of the last seven days, while the CNN model outperformed the LSTM model with a time window of the previous 13 days. Additionally, the CNN model required less execution time than the LSTM model.

Conclusions:

In conclusion, deep learning algorithms prove suitable for analyzing multivariate time series data. The CNN model demonstrated the lowest prediction error.

1. Background

Healthcare has emerged as one of the largest industries globally, with the emergency department (ED) being a vital component of these services and experiencing high demand (1). Overcrowding in hospital emergency departments has become a significant public health issue worldwide in recent decades, with ample evidence demonstrating its adverse effects, such as delays in treating critically ill patients, patient dissatisfaction, increased mortality, and higher rates of medical errors (2). The American College of Emergency Physicians defines ED crowding as a situation where the demand for emergency services surpasses the available resources for patient care in the emergency department, hospital, or both. Therefore, congestion occurs when the demand exceeds the supply of medical services, a challenge faced by various hospital departments due to their limited resources (3). Achieving optimal resource allocation in the healthcare sector and enhancing the overall health of society are crucial goals, albeit challenging, due to limited resources, high costs, and the sensitive nature of healthcare conditions (4). Consequently, determining the optimal volume of patients entering the ED is critical for effectively allocating resources. Past research has indicated that hybrid models may not necessarily outperform component models, highlighting the importance of exploring various reliable models with high predictive accuracy for forecasting the number of newly hospitalized patients using alternative data sources (5). Past studies have utilized meteorological and calendar information to estimate the rate of patient admissions to the ED, as these factors influence patient behavior upon entering the ED (6-8). Previous research has shown that the LSTM model outperforms other models in predicting hourly data, while simpler models have often been more effective than ensemble models in predicting the daily number of ED admissions (9). Another study focused on predicting the admission rate of patients with mental disorders, employing a combination of CNN and LSTM neural networks. It was found that the stacked ensemble of CNNs and LSTMs outperformed individual LSTM and CNN models (10). Additionally, previous research compared univariate and multivariate prediction models for forecasting ED patient arrivals during the COVID-19 epidemic, concluding that HW univariate modeling is suitable for short-term forecasting (11). Figure 1 illustrates the fundamental steps forming our research framework.

The main steps of the research framework
The main steps of the research framework
Figure 1.

The main steps of the research framework

2. Objectives

Based on the timing of data collection, our objective was to forecast the volume of patient admissions to the ED under both epidemic conditions like COVID-19 and non-epidemic scenarios. Furthermore, we aimed to compare the performance of the LSTM and CNN models.

3. Methods

3.1. Data Collection and Design

The collected data encompasses three main categories. The first category pertains to air pollutants, which were provided by the Tehran Air Quality Control Organization. The second type of data is associated with the Iran Meteorological Organization, and finally, we gathered information on the number of patients admitted to the ED of a hospital in Tehran (Table 1). In our datasheet, we also incorporated the binary representation of epidemic and non-epidemic conditions. The data collection period spans from February 2018 to March 2022. The selection of data types and predictors was informed by expert opinion and previous studies. We aggregated the averages of the predictors recorded by each air quality control station and the Iran Meteorological Organization. To facilitate a comprehensive review, we depicted the volume of ED patient admissions over four years in Figure 2.

Table 1.

Description of Features

ProviderFeaturesDescription
Meteorological organizationTemperatureMaximum temperature (tmax)
Minimum temperature (tmin)
Average temperature (tm)
Relative humidityMaximum relative humidity (umax)
Minimum relative humidity (umin)
Average relative humidity (um)
Wind and rainMaximum wind speed (ffmax)
Average wind speed (ffm)
Total rainfall during the last 24 hours (rr)
Air quality control organizationAir pollutantsCO, O3, NO2, SO2, PM10, PM2.5
Selected hospitalNumber of patients admitted to the EDInteger number
Ministry of HealthEpidemyBinary
The volume of patient admissions to the emergency department
The volume of patient admissions to the emergency department
Figure 2.

The volume of patient admissions to the emergency department

3.2. Data Preprocessing

The daily data collected over these four years exhibited no missing or duplicate values. An essential aspect to consider when analyzing time series data is whether it is stationary or non-stationary. A time series is considered stationary when its mean, covariance, and autocorrelation remain constant over time, meaning that observations do not depend on time. Predicting non-stationary time series is complex and often impossible (12). The Dickey-Fuller test, introduced by Dickey and Fuller in 1979, is used to assess the stationarity of time series data. The null hypothesis of this test assumes non-stationarity, while the alternative hypothesis suggests stationarity (13). Hence, we utilized the Dickey-Fuller test to evaluate the stationarity of our data.

To facilitate further analysis, features were normalized to fall within the range [0,1] using the min-max normalization method as defined in Equation 1 (14).

Equation 1.xi=xi-min⁡(xi)max⁡xi-min⁡xi

In this equation, xi represents the ith characteristic, while min (xi) and max (xi) denote the minimum and maximum values of Xi, respectively.

Subsequently, we partitioned the data into training and testing sets, allocating 80% of the data for training purposes. Finally, the feature vector comprised the current date's values of air pollutants and meteorological variables, along with the values of these variables from the previous \( N \) days. Additionally, the output values from the last N days were appended to the input feature vector.

3.3. Models

A subset of artificial intelligence (AI) methods known as machine learning learns the principles of interpretation from training data and applies them to new data within the same field to make intelligent decisions. Deep learning (DL) is a subset of machine learning that has recently garnered significant attention (15). One of the terms synonymous with DL is artificial neural networks (ANNs) (16). ANNs simulate the human brain, where neurons serve as processing units with inputs and outputs governed by activation functions (17). Analyzing multivariate weather time series data involves handling large datasets, making DL algorithms powerful tools for investigation (18). Long short-term memory (LSTM) is a robust recurrent neural system that represents an enhanced version of Recurrent Neural Networks (RNNs). Typically, when learning long-term dependencies, vanishing gradient problems may occur, which LSTM networks effectively address (19). LSTM units incorporate gates to discern which data in a sequence should be retained or discarded, including input, output, and forget gates, along with a cell with a self-reconnect connection. LSTM units are updated at each time step based on Equations 2 to 6.

Equation 2.ft=σg(wdztn-1+Ufzt-1N+bf)
Equation 3.it=σg(wiztN-1+ Uizt-1N+bi)
Equation 4.ot=σg(woztN-1+ Uozt-1N+bo)
Equation 5.ct=ftʘCt-1+itʘσc(wcztN-1+Uczt-1N+bc)
Equation 6.ztN=ot ʘσzct

Where ztN-1 and ztN represent the outputs of the LSTM unit at times N and (N-1), respectively. Ct determines the state of the LSTM unit, while U and b are weight matrix parameters trained during LSTM learning. Also, ft, it, and ot denote the outputs of the forget gate, input gate, and output gate, respectively. In addition, σg, σz, and σc refer to the sigmoid, hyperbolic tangent (tanh), and tanh activation functions, respectively. The indices of i, o, f, and c correspond to the input, output, forget, and state cells, respectively. The superscripts N-1, N, and g represent the LSTM unit input, output, and gate (10).

LSTM networks can overcome the limitations of traditional time series forecasting techniques by accommodating nonlinear data (20). Convolutional neural networks (CNNs) function similarly to traditional ANNs, comprising neurons optimized through self-learning (21). A noteworthy aspect of CNN networks is their utility for pattern discovery and extraction of horizontal relationships among multidimensional variables (22). The structure of CNN networks typically includes input layers, convolution layers, pooling layers for dimensionality reduction, and fully-connected layers (23). For model development, we independently designed LSTM neural networks and CNNs, considering various time windows of previous days. Figure 3 indicates the loss function for models.

Loss function per epochs chart for models
Loss function per epochs chart for models
Figure 3.

Loss function per epochs chart for models

3.4. Software

The Keras and TensorFlow libraries in Python software, along with the Google Colab environment, facilitated model execution. In terms of hardware, our system utilized an Intel i5 2.53 GHz CPU with 8 GB installed memory

4. Results

There were 319 880 patient arrivals to the ED between February 2018 and March 2022. Among these, 221 349 were male, and 98 531 were female. In the absence of a COVID-19 epidemic, the daily arrival of patients to the ED averaged 180 people. The highest number of arrivals during the COVID-19 epidemic conditions was 799. The root mean square error (RMSE) and mean absolute error (MAE) are widely used as standard statistical measures to evaluate model performance (24). Therefore, we utilized these two criteria to assess and compare the models. RMSE and MAE can be calculated using Equations 7 to 9.

Equation 7.RMSE=∑i=1n(pi-Ai)2n
Equation 8.MSE=(RMSE)2
Equation 9.MAE=∑i=1npi-Ain

Where Pi and Ai represent the predicted value and the actual value of the number of arrival patients to the ED for the ith test record, and for RMSE and MAE, smaller values are preferred. Another evaluation criterion is the MSE, which can be calculated using Equation 8.

Table 2 presents the evaluation of the models considering different time windows of past days. According to the table, LSTM had the lowest RMSE and MAE in the time window of the last seven days, while CNN performed even better than LSTM in the time window of the previous 13 days. To further monitor the models' performance on training and testing data based on the MSE criterion, we depicted the best performance of each model according to the time window of the previous day. Another noteworthy point is the execution time of building the model based on the training data. It took one minute and 23 seconds to construct CNN, while LSTM took two minutes and 24 seconds.

Table 2.

Evaluation Models by Considering Time Windows of Different Previous Days

Time Window, d
1471013
ModelsRMSEMAERMSEMAERMSEMAERMSEMAERMSEMAE
LSTM0.1510.1110.1360.1020.1300.0950.1420.1130.1350.105
CNN0.1620.1200.1420.1050.1220.0940.1290.0970.110.089

5. Discussion

Time series analysis is crucial in various fields, including finance, medicine, and aerospace, as forecasting demand has been appealing for suppliers. Time series data comprises measurements over time that depict system behavior (25). In medicine, predicting the rate of patient admissions to the ED aids managers in resource planning, which can mitigate crowding in this department. In this study, statistical analysis revealed positive correlations between air pollutants, temperature, epidemics, and the arrival rate, with epidemics and sulfur dioxide (SO2) exhibiting the highest correlations, respectively. Conversely, wind and humidity showed negative correlations. DL offers numerous advantages in automatically extracting and understanding multivariate data features, making it suitable for vast datasets with high computational requirements. Therefore, we employed DL algorithms like CNN and LSTM to analyze time series data and predict future trends.

However, this study has several limitations. Firstly, we only examined data from one hospital in Tehran. By making slight adjustments to the prediction models and incorporating data from other hospitals, we can generalize the models to EDs in different hospital settings. Secondly, the study's results are limited to our geographic area's climatic characteristics, emphasizing the need for caution when applying these findings to hospitals in other geographic regions.

Also, we did not consider snowy weather conditions in this study. Additionally, the study did not evaluate patient grouping based on diagnosis or severity. Given the resource constraints in the ED, effective resource management is vital. Accurately predicting patient arrivals can significantly improve resource allocation and management in the ED. In future studies, we aim to employ a combined CNN and LSTM model known as a stack ensemble to predict hourly patient arrivals to the ED and identify peak arrival hours

5.1. Conclusions

This study concludes that DL algorithms are well-suited for analyzing multivariate time series data, with the CNN model exhibiting the lowest prediction error. The study indicates a positive correlation between air pollutants and ED patient arrival rates, suggesting that policy interventions should target reducing air pollutants to mitigate ED overcrowding, particularly in the absence of COVID-19.

References

  • 1.

    Sariyer G, Öcal Taşar C, Cepe GE. Use of data mining techniques to classify length of stay of emergency department patients. Bio-Algorithms Med-Syst. 2019;15(1). https://doi.org/10.1515/bams-2018-0044.

  • 2.

    Hsu CM, Liang LL, Chang YT, Juang WC. Emergency department overcrowding: Quality improvement in a Taiwan Medical Center. J Formos Med Assoc. 2019;118(1 Pt 1):186-93. [PubMed ID: 29665984]. https://doi.org/10.1016/j.jfma.2018.03.008.

  • 3.

    Schoenenberger LK, Bayer S, Ansah JP, Matchar DB, Mohanavalli RL, Lam SS, et al. Emergency department crowding in Singapore: Insights from a systems thinking approach. SAGE Open Med. 2016;4:2050312116671950. [PubMed ID: 27757231]. [PubMed Central ID: PMC5052930]. https://doi.org/10.1177/2050312116671953.

  • 4.

    Keshtkar L, Salimifard K, Faghih N. A simulation optimization approach for resource allocation in an emergency department. QSci Connect. 2015;2015(1). https://doi.org/10.5339/connect.2015.8.

  • 5.

    Zhou L, Zhao P, Wu D, Cheng C, Huang H. Time series model for forecasting the number of new admission inpatients. BMC Med Inform Decis Mak. 2018;18(1):39. [PubMed ID: 29907102]. [PubMed Central ID: PMC6003180]. https://doi.org/10.1186/s12911-018-0616-8.

  • 6.

    Calegari R, Fogliatto FS, Lucini FR, Neyeloff J, Kuchenbecker RS, Schaan BD. Forecasting Daily Volume and Acuity of Patients in the Emergency Department. Comput Math Methods Med. 2016;2016:3863268. [PubMed ID: 27725842]. [PubMed Central ID: PMC5048091]. https://doi.org/10.1155/2016/3863268.

  • 7.

    Jones SS, Thomas A, Evans RS, Welch SJ, Haug PJ, Snow GL. Forecasting daily patient volumes in the emergency department. Acad Emerg Med. 2008;15(2):159-70. [PubMed ID: 18275446]. https://doi.org/10.1111/j.1553-2712.2007.00032.x.

  • 8.

    Kam HJ, Sung JO, Park RW. Prediction of Daily Patient Numbers for a Regional Emergency Medical Center using Time Series Analysis. Healthc Inform Res. 2010;16(3):158-65. [PubMed ID: 21818435]. [PubMed Central ID: PMC3089856]. https://doi.org/10.4258/hir.2010.16.3.158.

  • 9.

    Zhang Y, Zhang J, Tao M, Shu J, Zhu D. Forecasting patient arrivals at emergency department using calendar and meteorological information. Appl Intell (Dordr). 2022;52(10):11232-43. [PubMed ID: 35079202]. [PubMed Central ID: PMC8776398]. https://doi.org/10.1007/s10489-021-03085-9.

  • 10.

    Khatibi T, Karampour N. Predicting the number of hospital admissions due to mental disorders from air pollutants and weather condition descriptors using stacked ensemble of Deep Convolutional models and LSTM models (SEDCMLM). J Clean Prod. 2021;280:124410. https://doi.org/10.1016/j.jclepro.2020.124410.

  • 11.

    Etu EE, Monplaisir L, Masoud S, Arslanturk S, Emakhu J, Tenebe I, et al. A Comparison of Univariate and Multivariate Forecasting Models Predicting Emergency Department Patient Arrivals during the COVID-19 Pandemic. Healthcare (Basel). 2022;10(6). [PubMed ID: 35742171]. [PubMed Central ID: PMC9222821]. https://doi.org/10.3390/healthcare10061120.

  • 12.

    Manuca R, Savit R. Stationarity and nonstationarity in time series analysis. Physica D: Nonlinear Phenomena. 1996;99(2-3):134-61. https://doi.org/10.1016/s0167-2789(96)00139-x.

  • 13.

    Jalil A, Rao NH. Time Series Analysis (Stationarity, Cointegration, and Causality). Environmental Kuznets Curve (EKC): A Manual. Academic Press; 2019. p. 85-99. https://doi.org/10.1016/b978-0-12-816797-7.00008-4.

  • 14.

    Han J, Kamber M, Pei J. Data Preprocessing. Data Mining. Morgan Kaufmann; 2012. p. 83-124. https://doi.org/10.1016/b978-0-12-381479-1.00003-4.

  • 15.

    Chassagnon G, Vakalopolou M, Paragios N, Revel MP. Deep learning: definition and perspectives for thoracic imaging. Eur Radiol. 2020;30(4):2021-30. [PubMed ID: 31811431]. https://doi.org/10.1007/s00330-019-06564-3.

  • 16.

    Goodfellow I, Bengio Y, Courville A. Deep learning. MIT Press; 2016.

  • 17.

    Kukreja H, Bharath N, Siddesh CS, Kuldeep S. An introduction to artificial neural network. Int J Adv Res Innov Ideas Educ. 2016;1:27-30.

  • 18.

    Zhang Z. Artificial Neural Network. Multivariate Time Series Analysis in Climate and Environmental Research. Springer; 2018. p. 1-35. https://doi.org/10.1007/978-3-319-67340-0_1.

  • 19.

    Van Houdt G, Mosquera C, Nápoles G. A review on the long short-term memory model. Artif Intell Rev. 2020;53(8):5929-55. https://doi.org/10.1007/s10462-020-09838-1.

  • 20.

    Chimmula VKR, Zhang L. Time series forecasting of COVID-19 transmission in Canada using LSTM networks. Chaos Solitons Fractals. 2020;135:109864. [PubMed ID: 32390691]. [PubMed Central ID: PMC7205623]. https://doi.org/10.1016/j.chaos.2020.109864.

  • 21.

    O'Shea K, Nash R. An introduction to convolutional neural networks. arXiv preprint arXiv:1511.08458. 2015. https://doi.org/10.48550/arXiv.1511.08458.

  • 22.

    Jin X, Yu X, Wang X, Bai Y, Su T, Kong J. Prediction for Time Series with CNN and LSTM. Proceedings of the 11th International Conference on Modelling, Identification and Control (ICMIC2019). Springer; 2020. p. 631-41.

  • 23.

    Wu J. Introduction to convolutional neural networks. National Key Lab Novel Soft Technol. 2017;5(23):495.

  • 24.

    Chai T, Draxler RR. Root mean square error (RMSE) or mean absolute error (MAE)? – Arguments against avoiding RMSE in the literature. Geosci Model Dev. 2014;7(3):1247-50. https://doi.org/10.5194/gmd-7-1247-2014.

  • 25.

    Aminikhanghahi S, Cook DJ. A Survey of Methods for Time Series Change Point Detection. Knowl Inf Syst. 2017;51(2):339-67. [PubMed ID: 28603327]. [PubMed Central ID: PMC5464762]. https://doi.org/10.1007/s10115-016-0987-z.