Study on the Efficiency of a Multi-layer Perceptron Neural Network Based on the Number of Hidden Layers and Nodes for Diagnosing Coronary- Artery Disease

1. Background

Artificial intelligence is used for the development of algorithms, which are to be used by computers (1). One of the essential requirements for any type of intelligent behavior learning and machine learning is research in artificial intelligence, which is expanding rapidly (2). Among the current data-mining methods and machine-learning methods is the application of an artificial neural network that has a high level of predictability (3). Due to its ability to perform a parallel processing and to create a distributed storage, in addition to its high degree of fault tolerance, an artificial neural network is capable of increasing efficiency (4). An artificial neural network is also capable of analyzing, modeling, and comprehending complex medical information on a wide range of applied programs and is, therefore, an essential tool for medical practitioners (5). Thus, it meets the needs of the medical practitioner for reliable and much needed medical diagnosis (6). Among the most common artificial neural networks used in the majority of medical research programs is the multi-layer perceptron (MLP) artificial neural network (7). This type of neural network is trained through the application of backpropagation algorithms and is used mostly for diagnostic decision support systems (8). The number of layers and hidden nodes describes the structure and architecture of the MLP artificial neural network. The MLP network with one or more hidden layers is renowned globally as a predictive program (9). The global predictive theory informs neural networks that any continuous function that maps the same range of cardinal values in the range of its cardinal value output can only be predicted accurately using an MLP with one hidden layer (10).

A neural network with one hidden layer can map out anything it requires. In practice, two hidden layers are used to increase the speed of convergence while no further hidden layers are required. The designer of the network should be able to solve an equation with one or two hidden layers (11). The number of nodes in the hidden layers will definitely affect the capability of the neural network to generate inaccessible data from the learned data. Neural networks with few hidden nodes are not suitable for modeling purposes in the training phase, while a neural network with many hidden nodes might determine the decisive boundaries in the vector space, which in themselves are strongly affected by the specific characteristics of the learned data (12). The neural network is sensitive to the number of neurons in its hidden layers. The limited number of neurons will result in non-adjustment and the high number of neurons will result in tight fitting. This essentially means that the learned data will fit appropriately, yet the fitness value curve will fluctuate wildly among these points (13). Based on the recommendation of the developers in Table 1, various relationships have been introduced for calculating the neurons in hidden layers (14).

Nowadays, the golden standard for evaluating the constriction or obstruction of coronary arteries is angiography, which is widely common (15). Most of the cases that are introduced for t angiography to determine the obstruction of the coronary artery fail to have any such obstruction. Yet, negative results from the test are sometimes inevitable because non-aggressive methods, which are used prior to angiography, are not very sensitive and specific. In addition, the patient, the Physician, and the law that acceptance of a low percentage of false positive results is illogical when compared with the number of negative diagnoses do not allow the patient’s life to be endangered. In order to decrease the number of false positive reports, several methods have been suggested as being used as a pre-evaluation for angiography (16). The first point to be considered here is that it is possible to use data mining techniques in order to diagnose patients suffering from coronary artery disease. Using an MLP neural network, one can obtain important results related to the diagnosis of coronary artery disease. The results of the current research show that the greater the accuracy of the results obtained from the MLP approach, the lower the number of patients not actually suffering from a coronary artery disease subjected to unnecessary angiography (which is in itself an invasive and potentially life-threatening procedure). What is more, greater sensitivity in the obtained results detracts from unnecessary costs and waiting time for patients requiring results (17).

Various models have been developed in various studies to diagnose coronary artery disease, using different multi-layer perceptron (MLP) neural networks, leading to different results with different variables and conditions. For instance, in a research, changing the number of risk factors and genotypes of the disease, various models of MLP with two hidden layers and 4 - 4 hidden nodes were created that caused a diagnosis with different accuracies from 64% to 93% (18). In another study, the MLP neural network was trained with various learning algorithms, reporting that the best state of the network was with one hidden layer and seventeen secret nodes, leading to 96% sensitivity, 91% specificity, and 87% accuracy (19). The study of Tsipouras et al. was conducted with a dataset of 199 cases and 19 specificities including demographic data, disease history, and laboratory findings to diagnose the coronary artery disease. In this MLP artificial neural network, one hidden layer and ten hidden nodes were considered and a sensitivity of 80%, a specificity of 66.3%, and an accuracy of 73.9% in the diagnosis of the disease were provided (20). Stefko’s research on the ECG results was carried out in order to diagnose the coronary artery disease using a multilayer perceptron neural network with the back-propagation algorithm. By the use of a network including one hidden layer and five hidden nodes, a sensitivity of 100% and a specificity of 75.83% for the diagnosis of the disease were obtained (21). According to the previous discussion, applying the artificial intelligence techniques in diagnostic decision support systems can prevent the use of a dangerous angiography by early diagnosis of the disease. This study aimed to evaluate the efficiency of multi-layer perceptron artificial neural network technique based on the number of hidden layers and hidden nodes in order to diagnose the coronary artery disease. It is hoped that by presenting a high-efficient model, the risks and costs of diagnosis would be declined.

2. Methods

In this fundamental analytical research, the database of the Tehran cardiac center was utilized and 13,945 records related to nine risk factors including age, gender, body mass index (BMI), body fat, family history, substance dependency (smoking), blood cholesterol, diabetes, and high blood pressure were taken into account. The descriptive statistics related to the nine risk factors were incorporated in Tables 2 and 3. Using Statistica version 10 software and through the application of classification and regression tree (CART), Table 4 which shows the ranking and level of importance of the risk factors was obtained.

The database was then prepared using SPPS 17. 717 records were omitted due to missing values and the number of records used in the database declined to 13,228.

Considering the fact that, in this research, a person who was suffering from the stenosis of more than 50% in at least one vessel was considered as a patient, the number of healthy people and patients was respectively recorded as 4019 and 9209 in the database. In other words, 30.38% of the records were related to healthy people and 69.62% belonged to patients, and as a result, the number of classes for 4019 healthy people and 9209 patients was not balanced. A dataset is unbalanced when the number of classification categories is almost unequal. Applying over-sampling methods in an abnormal class (smaller class) and under-sampling in a normal class (larger class), a better performance can be achieved in the classification (22).

It is of note that during the pre-processing phase, new data might be added to the database (over-sampling) or data might be omitted from the database (under-sampling). Thus, in this research, synthetic minority over-sampling technique (SMOTE) was applied which uses artificial samples in the over-sampling of the smaller group. In this method, in addition to a greater amount of time spent on the training of the neural network, based on the selection of the characteristic, a sub-category of the characteristic was selected and applied in the classification with a higher number using a ranking filter. This resulted in a higher efficiency in the classification of the data (23). By applying the SMOTE to the data, the total number of data increased to 16,076, of which 8038 cases were considered as patients and 8038 as healthy individuals.

Using MATLAB, binary codes were applied to the variables of gender and substance abuse (smoking) using dummy variables (due to the fact that the relationship between both variables was unreal) (24).

Based on the laws governing hidden layers and nodes in MLP neural networks, in this study, the maximum number of hidden layers did not exceed two layers and the maximum number of hidden neurons did not exceed 27 neurons (3 times the number of neurons in the input nodes). By including dummy variable, the maximum number of neurons in each hidden layer remained at 36 (14). As a result, the various models of the MLP neural network were consisted of 9 input nodes which in themselves contained 2 hidden layers, consisting of 1 to 36 hidden nodes in each layer, and produced one output layer, consisting of one node.1332 MLP neural network models were created.

The data were then randomly divided into three categories: a training category (%70), an evaluation category (%15), and a test category (%15). The evaluative category was used to determine the internal validity of the model and prevent overfitting. In order to obtain the highest level of sensitivity possible, and to determine the specifications and accuracy of the model so that the model’s efficiency can be evaluated, a perfcurve function was used. This function determines the area beneath the receiver operating characteristic (ROC) curve and calculates the amount of sensitivity, specificity, and accuracy of the models. The ROC curve is a two-dimensional curve. Its vertical axis is the correct detection rate of the positive category and its horizontal axis is the wrong detection rate of the negative category. In fact, the ROC chart shows the relative compromise of profits and costs (25). In the final stage, 1332 different models of MLP neural network were trained and tested by changing the number of hidden nodes and hidden layers.

Sensitivity, specificity, and accuracy are defined as follows:

Sensitivity = TP/(TP + FN) = (Number of true positive assessment)/(Number of all positive assessment)

Specificity = TN/(TN + FP) = (Number of true negative assessment)/(Number of all negative assessment)

Accuracy = (TN + TP)/(TN + TP + FN + FP) = (Number of correct assessments)/Number of all assessments)

Sensitivity is the proportion of true positives that are correctly identified by a diagnostic test. It shows how good the test is at detecting a disease. Specificity is the proportion of true negatives correctly identified by a diagnostic test. It suggests how good the test is at identifying normal (negative) condition. Accuracy is the proportion of true results, either positive or negative, in a population. It measures the degree of veracity of a diagnostic test in a condition (26).

The term of accuracy points to the few false positives (FPs) but the efficiency points to the few missed positives. The most popular and useful tools for efficiency assessments are methods based on the receiver operating characteristic (27).

3. Results

In the current research, the efficiency of the models was compared based on the number of hidden layers and hidden nodes in them using a ROC curve analysis. In these models, the range of variation in the ROC curve area was between 0.69 and 0.82. The range of variations in sensitivity was also between 0.66 and 0.92 and the range of variations in specificity was 0.53 to 0.75 while the range of variations in accuracy was between 0.69 and 0.82.

The characteristics of the models with the highest values of sensitivity, specificity, accuracy, and the surface area under ROC curve are summarized in Table 5. We explain each one in the following.

Considering the surface below ROC curve as a criterion for the efficiency of the models, the best model with the highest surface ROC of 0.82 was selected. This model had two hidden layers; there were 34 hidden neurons in the first hidden layer and 18 hidden neurons in the second hidden layer. In this model, the sensitivity of 0.90, specificity of 0.73, and accuracy of 0.82 were obtained. In this model, the rate of specificity was less than that of sensitivity; therefore, it can be said that the model is more effective in identifying patients than in identifying healthy individuals.

According to the highest rate of sensitivity index, the model of the MLP neural network had two hidden layers including 24 hidden neurons in the first layer and 23 hidden neurons in the second layer. This model took the shortest time of network training. In this model, the sensitivity of 0.92, specificity of 0.70, and accuracy of 0.81 were calculated. The rate of specificity was less than that of sensitivity, indicating the mode is more potent in identifying patients than in identifying healthy people.

In terms of the highest rate of specificity, a model of the MLP neural network with two hidden layers, including 24 hidden neurons in the first layer and 36 hidden neurons in the second layer with a specificity of 0.75, was selected. In this model, sensitivity was 0.88 and accuracy was 0.82. Using this model, the system is more capable of identifying patients than identifying healthy people.

Moreover, due to the highest rate of accuracy, a model of the MLP neural network with two hidden layers including 34 hidden neurons in the first layer and 18 hidden neurons in the second layer, with an accuracy of 0.82, was selected. In this model, sensitivity was 0.90 and specificity was 0.75. This model had the highest ratio of correct detection to all diagnostics among other models. In this model, the system was also more capable of identifying patients than identifying healthy people.

4. Discussion

In this research, the efficiency of MLP model of the artificial neural network, based on the number of hidden layers and hidden nodes in diagnosing the coronary artery disease, was studied. The difference in available data, different variables, the difference in the rank of risk factors in Iran, as well as the lack of access to the exact details of the models, led to the construction of new models matched to the data of the heart center of Tehran. The risk factors of the disease (age, sex, BMI, abdominal obesity, family history, smoking, high blood glucose, diabetes, and high blood pressure) were considered as the input of the neural network, and different network models with the change in the number of layers and hidden nodes were created.

Regarding the area under the ROC curve, the model that contained 2 hidden layers with 34 and 18 hidden nodes and 0.82 area under the ROC curve had the best efficiency in diagnosing the disease.

Various studies have been conducted by other researchers using the MLP model in order to diagnose the coronary artery disease, in which other risk factors were used. Some of these studies are presented in Table 6 to be compared with each other.

In a research conducted by Atkov et al., a set of genetic and non-genetic factors was used. With the change in the number of risk factors and genotypes of the disease, different MLP models with 2 hidden layers and 4 - 4 hidden nodes were created, which determined the range of accuracy from 64% to 93% (18). The research conducted by Atkov et al. differed from the present study in the number and type of risk factors, the number of database records, and the number of hidden nodes. The accuracy of Atkov’s and colleagues’ research was higher than that of the current research and also its ability in an accurate diagnosis was higher than that of our research, which can be attributed to the empowerment of the model due to the high number of input risk factors. However, in that research, the risk factors and genetic factors that were used to diagnose the disease required specialized diagnostic tests; while in the present study, the most basic risk factors that are measurable at the earliest level of services and therefore consume less cost and time were used.

In the study of Colak et al., the MLP neural network was trained with various learning algorithms (19). The number of risk factors in this study was more than that of the current research, and the number of dataset records was far less than that in the present study. Compared to the present research, this study had a higher rate of specificity; in other words, it was more potent in identifying healthy people, which could be due to the higher number of risk factors involved in the model. However, in both studies, with a slight difference, sensitivity and accuracy were similar. Therefore, both have the same ability in the accurate diagnosis of patients, which can be due to the use of approximately the same type of risk factors.

In the study of Tsipouras et al., the rule-based decision support system and MLP artificial neural network were investigated for the diagnosis of coronary artery disease. The input dataset included demographic data, disease history, and laboratory findings (20). In the study conducted by Tsipouras et al., there were more risk factors but fewer information records compared to the present research. The models were also different from each other in terms of the number of hidden layers and hidden nodes. In the research of Tsipouras et al., sensitivity, specificity, accuracy, as well as efficiency were less than those in the present study. The reason for the higher efficiency of the model in this study can be due to the preprocessing methods as well as the higher number of information records and consequently a greater number of training sets in the MLP neural network.

In Stefko’s research, a multilayer perceptron neural network with back-propagation algorithm was used to diagnose coronary artery disease. The research on ECG results was based on the results of coronary angiography (21). Stefko’s research and the present study differed in the type of MLP network input. Moreover, its sensitivity was higher than that of the present study; however, they were almost the same in terms of their specificity. In the present study, the risk factors of the disease were used as network input that can easily be measured in health centers and it was not required testing ECG for diagnosis of the disease.

In the study conducted by Chong et al., the artificial neural network model was used to predict the major side effects in patients experienced on-pump coronary artery bypass grafting surgery. The CABG (coronary artery bypass grafting) database, including 563 patients, was used and the ability of the MLP artificial neural network was evaluated considering the area under ROC curve (28). The number of network inputs in Chong et al.’s research was greater than that of the present research. The neural network presented in these two studies differed in terms of the number of hidden layers and hidden nodes. However, both networks provided an almost similar sensitivity and specificity.

In a research conducted by Fujita et al., the MLP network was used in the images of the SPECT Bull’s-eye heart for the diagnosis of coronary artery disease. The MLP artificial neural network was used with the back-propagation algorithm and one hidden layer. Then, the network was trained with 5 to 140 hidden nodes that, in the best situation, with 100 hidden nodes, caused 77% correct diagnosis (29). The number of hidden layers and hidden nodes in the present study was more than those of Fujita’s research and colleagues, and it also had higher accuracy. The advantage of this study is the diagnosis of the disease with the help of the simplest risk factors of the disease that are much easier to access than the Myocardial SPECT Bull’s-eye Images. In addition, the higher detection capacity of the disease in this research is due to the neural network architecture as well as the number and type of training data in the network.

Comparing the results of the previous research with those of the present study, it can be concluded that the differences in the number and type of risk factors, the number of database records as training data of the MLP neural network, the number of hidden layers, and the number of hidden nodes in the network, causing different sensitivities, specificities, and accuracies. In addition, in most studies mentioned above, ROC curve analysis was not used to evaluate the model’s efficiency and only three factors of sensitivity, specificity, and accuracy were reported. However, in this study, using the area under ROC curve, the ability of the MLP neuronal network model to identify healthy and unhealthy people could be increased to 0.82. That is why ROC curve is a suitable criterion to provide the highest rate of efficiency among the models in diagnosing the coronary artery disease.

Therefore, it is recommended that all of the risk factors for the disease, laboratory findings, and diagnostic tests in health centers be recorded in order to provide models with higher efficiency in further research.

4.1. Conclusion

By changing the number of hidden layers and hidden nodes, a different range of MLP neural network efficiency for diagnosing the coronary artery disease is obtained. Therefore, the proposed model in the present study, with regard to the surface under the ROC curve, provided the highest rate of efficiency and relatively acceptable results. This advantage, despite the low number of risk factors and the diagnostic methods used in this study compared to other studies, can be due to data preparation and processing methods. Based on the use of the fewest available variables and easy access to these risk factors with the lowest cost at the lowest level of healthcare provision as well as the lack of need for specialized diagnostic tests, the research findings indicate that the MLP neural network is an acceptable approach to diagnose coronary artery disease. Given that the specificity of the selected model is less than its sensitivity, the system is more capable of distinguishing patients than identifying the healthy people; however, it has also an acceptable performance in identifying healthy people. Accordingly, it can be stated that the models of neural network generated from this study are capable of helping making algorithm in medical decision support systems to diagnose the disease; thus, they can be replaced by the invasive and dangerous method of angiography in future.