Detection of Neonatal Jaundice by Using an Android OS-Based Smartphone Application

1. Background

Neonatal jaundice, presenting as yellow discoloration of the skin, sclera, and mucous membranes in infants and indicating raised serum bilirubin level, is one of the most common clinical conditions that may need medical attention (1). Unconjugated hyperbilirubinemia is the most common form of hyperbilirubinemia observed in neonates (2). Neonatal jaundice generally appears around two to four days after birth and vanishes one to two weeks later, often without the need for treatment. In addition, jaundice constitutes a common cause of hospital re-admission after early discharge of newborns (3). Cumulatively, 24 million newborns develop jaundice every year (4). Approximately, 80% of preterm and 60% of term infants develop jaundice in the first week of life (2).

Unconjugated bilirubin can penetrate the blood-brain barrier in newborns and is potentially neurotoxic. It can cause acute or chronic encephalopathy, also known as kernicterus, that may lead to undesirable neurological outcomes like cerebral palsy, hearing loss, and seizures (3).

The major challenge is to differentiate physiologic neonatal jaundice, which is harmless, from pathologic jaundice, which might lead to kernicterus and even death (1). Total serum bilirubin (TSB) level generally peaks in newborns at approximately 96 hours of life, which is well after most infants are discharged (5).

To determine whether a newborn should receive proper medical treatments like phototherapy and exchange transfusion, medical professionals reference specialized graphs, such as Bhutani nomogram, with the newborn’s age, number of weeks of gestation, and bilirubin level. Based on Bhutani nomogram, which provides a means to assess a newborn’s risk of pathological neonatal jaundice, high intermediate risk is considered above the 75th and high risk above the 95th percentile (6).

In order to initiate appropriate management methods that can both prevent and treat severe neonatal jaundice, screening methods that measure bilirubin level are needed (7). The gold standard for determining the level of hyperbilirubinemia is TSB measurement, which is performed by blood sampling from newborns (8). However, the measurement of TSB is an invasive and stressful procedure that can only be done by medical caregivers. Also, detecting the level of hyperbilirubinemia by measuring TSB can result in blood loss, an increased risk of infections at the site of sampling, and increased anxiety in parents (9, 10).

2. Objectives

Simple-to-use tools that could facilitate early detection of neonates with pathologic jaundice can hinder the development of bilirubin-induced neurodevelopmental sequelae. To achieve this goal, we designed an easy-to-use Android-based application to detect neonatal jaundice.

3. Methods

3.1. Enrollment and Data Collection

To evaluate the design of our smartphone-based neonatal jaundice detection system, we conducted a prospective two-center clinical study at Hafez and Shoushtari hospitals, in Shiraz, Iran, and created a dataset of image samples paired with bilirubin levels from TSB tests. We captured the images of the participant neonates within less than 5 minutes of the TSB blood draw to be certain that bilirubin levels were as accurate as possible. The participants were healthy, ≤ 9 days old, and born at ≥ 35 weeks’ gestation. Neonates who had received phototherapy were not considered in this study. A total of 113 neonates were enrolled in our study. Basic demographic features of the participants are listed in Table 1. Parents of the newborns gave informed consent to participate in the study. Each newborn was assigned a distinctive study ID to match the participant’s TSB measurement results and image data while maintaining confidentiality.

The TSB assays at Shoushtari and Hafez hospitals were run by using Beckman AU680 platform and Bilimeter 3 (Pfaff Medical), respectively.

3.2. Description of the Technology

Our application was designed to acquire images of the newborn’s forehead skin in a standardized manner; the application uses offline machine learning and regression techniques for analysis. All the images were collected by medical professionals using the built-in camera on your Samsung Galaxy Grand Prime, a 100X zoom microscope (cellphone magnification clip), and a custom-designed data collection app. (Figure 1).

Sample imaging technique that was used in our study

Figure 1.

Sample imaging technique that was used in our study

To obtain a set of images, a color calibration card is placed on the newborn’s forehead. The calibration card contains red, green and blue (R, G, B) colors, which are printed on an ~ 8 cm by 6 cm paper. The R, G, B components of the calibration card are converted by a digital cannon printer to C, M, Y, K (cyan, magenta, yellow, K = black) (Figure 2). The user places the 100X microscope on the built-in camera of the phone. The focus of the microscope could be changed to achieve a vivid image on the smartphone monitor. Also, there is a white light-emitting diode (LED) in our microscope with the intensity of 550lux that affects the monotony of light reflected on the sampling area.

Design of the color calibration card we used in our study

Figure 2.

Design of the color calibration card we used in our study

By using the custom application, the user should take five images from each color of the calibration card (R, G, B), which results in a total of 15 images. Following this step, the user takes five images from the forehead skin of the newborn. The overall process of obtaining the images typically takes less than 60 seconds.

The application analyzes all the images pixel by pixel and estimates the average R, G, B scores of the images taken from the forehead skin and calibration card. Then, RGB parameters are converted to hue, saturation, intensity (HSI) parameters, and the application calculates the R, G, B, H, S values using four different formulas, namely X1, X2, X3, X4. All these steps are offline and do not require internet connection.

3.3. Algorithm Development

Our application, which is programmed by Eclipse with upgraded adt 2014, is designed for Android OS (operating system) and uses smartphone camera API (application programming interface). Also, by setting the variables and calling the functions of the objects , smartphone-taken images can be analyzed pixel by pixel. The images taken with the smartphone camera contain sets of biometric features that can be used for the classification of the images. The classification of the images answers the question of how can all the sub-collections, which include the biometric characteristics, be identified? The answer to this question is specified in our flowcharts (Figures 3 and 4).

Flowchart of neonatal jaundice detection application for analysis of calibration card image

Figure 3.

Flowchart of neonatal jaundice detection application for analysis of calibration card image

Flowchart of neonatal jaundice detection application for analysis of forehead skin image

Figure 4.

Flowchart of neonatal jaundice detection application for analysis of forehead skin image

By exerting machine learning and its subcategories such as supervised learning and regressive methods, the application recognizes sets of unique biometric features that highly correlate with TSB levels. Biometric characteristics from each of the 20 images acquired from the study participants were then used to select a final set of biometric features. These characteristics were included in our regression models to estimate bilirubin levels.

The 100X zoom microscope used in our study gave us a circular space with a radius of approximately 1200 pixels. To take a distance from the edge of this circle and avoid noises associated with the edge of the circle, we created a rectangle with a width of 1100 pixels and a length of 1300 pixels in this circular space and performed our pixel by pixel analysis algorithm in this rectangular space (Figure 5).

To avoid the noise associated with the edge of the circular field of view of the 100X microscope, we created a rectangle with a width of 1100 and a length of 1300 pixels in this circular field and performed pixel by pixel analysis algorithm in this rectangular space

Figure 5.

To avoid the noise associated with the edge of the circular field of view of the 100X microscope, we created a rectangle with a width of 1100 and a length of 1300 pixels in this circular field and performed pixel by pixel analysis algorithm in this rectangular space

4. Results

A complete set of images were obtained from the 113 participants along with their corresponding TSB level. The mean age of the participants at the time the images were taken was three days, with a range of 2 to 9 days. The mean TSB level in the participants was 6.82 mg/dL, with the range of 2.7 mg/dL to 19.3 mg/dL.

All the data gathered from the analysis of each newborn set of images and their matching TSB values were entered into SPSS for statistical analysis.

To predict the exact level of bilirubin estimated by our application, we divided the bilirubin levels into two subsections: The first subsection was for TSB levels of less than 10 mg/dL with their matching app-estimated bilirubin and the second subsection was for TSB levels of less than 15 mg/dL with their matching app-estimated bilirubin. Then, we used four different formulas or methods, namely X1, X2, X3, X4, each of which has it exclusive Pearson correlation coefficient, sensitivity, and specificity to estimate the level of image-based bilirubin.

The matching Pearson correlation coefficients and P values of X1 to X4 formulas are described in Table 2.

We used X1 in our final application because X1 had the highest Pearson correlation (0.479) with the TSB values. X1 could be formulated in a linear function, in which X1 is the image-based estimation of bilirubin:

X1 = 1.134 × x + 2.685

X1 has a sensitivity of 68% and specificity of 92.3% for estimating the bilirubin levels of less than 10 mg/dL and sensitivity of 82.1% and specificity of 100% for estimating the bilirubin levels of less than 15 mg/dL. X1 has positive likelihood ratio of 8.83 and negative likelihood ratio of 0.35 for bilirubin levels of less than 10 mg/dL and negative likelihood ratio of 0.18 for bilirubin levels of less than 15 mg/dL.

The corresponding sensitivity and specificity of all our formulas are summarized in Figure 6 with receiver operating characteristic curve (ROC curve) for each formula. (Figure 6).

Roc curve for neonatal jaundice application. A, sensitivity and specificity corresponding to X1; B, sensitivity and specificity corresponding to X2; C, sensitivity and specificity corresponding to X3; D, sensitivity and specificity corresponding to X4. e stands for bilirubin level of less than 10, and f stands for bilirubin level of less than 15.

Figure 6.

Roc curve for neonatal jaundice application. A, sensitivity and specificity corresponding to X1; B, sensitivity and specificity corresponding to X2; C, sensitivity and specificity corresponding to X3; D, sensitivity and specificity corresponding to X4. e stands for bilirubin level of less than 10, and f stands for bilirubin level of less than 15.

5. Discussion

The results of the statistical analysis of our smartphone-based bilirubin estimation show that it can provide a reasonable estimation of bilirubin level in newborns. Our smartphone-based bilirubin estimation can serve as a screening tool to determine neonates requiring blood sampling to measure TSB.

There have been some previous studies on the use of other noninvasive methods to predict bilirubin levels in neonates. All babies should be visually inspected for jaundice at every opportunity using Kramer’s scale, which describes the relationship between serum bilirubin level and the progression of skin discoloration (8).

Another method to detect the level of bilirubin in neonates is the use of transcutaneous bilirubinometer (TcB), a hand-held device that measures the amount of bilirubin in the skin by directing light into the skin of the newborn and evaluating the intensity of specific wavelengths returned. TcB is a bed-side non-invasive rapid screening test that can be used to estimate total serum or plasma bilirubin and thus, avoid blood sampling. These devices have been shown to correlate well with serum bilirubin levels in term and near-term infants. However, mean differences between the hyperbilirubinemia level measured by TcB devices and TSB levels are large when the bilirubin levels exceed 205 μmol/L (12 mg/dL) in neonates. Another setback of transcutaneous meters is that they are rarely available in low- and middle-income countries. Also, there is conflicting evidence with regards to the sensitivity of TcB usage in pigmented neonates (7, 8, 11, 12).

BiliSpec is a low-cost reader and disposable lateral flow card that needs several drops of blood to determine the concentration of total bilirubin (4). Icterometer, a screening tool that can be used to detect the level of hyperbilirubinemia, is comprised of four yellow strips of paper that are pressed on the nose of neonates to predict bilirubin concentration (13). Bilistrip™, a two-color icterometer, uses the same method to determine the level of hyperbilirubinemia (14).

An additional potential screening technique for assessing neonatal jaundice exploits the yellow discoloration of the sclera by obtaining digital photographs of the newborn’s eyes and processing the pixel color values of the sclera to estimate TSB levels. This method can identify neonates with TSB levels above 205 µmol/L with a sensitivity of 1.00 and specificity of 0.50. One of the drawbacks of this method is the influence of ambient light and room light on the estimated bilirubin level (15).

In another study by Aydin et al., a non-invasive smartphone-based system for the detection of neonatal jaundice was devised. They employed advanced image processing techniques and machine learning regression to process images taken from 40 jaundiced newborns and achieved the correlation rate of 85% with TSB (16).

Taylor et al. designed an IOS-based smartphone application called BiliCam that analyses digital images taken with a smartphone (iPhone 5s) to estimate the level of hyperbilirubinemia. This process in initiated by placing a color calibration card on the neonate’s sternum, and then by using the BiliCam application, a set of standardized images are taken and transmitted via the Internet to a computer server for analysis. For designing BiliCam, Taylor et al. used two separate decision rules to predict the level of newborn’s bilirubin. The sensitivities of BiliCam in recognizing neonates with high TSB levels were 84.6% and 100%, respectively, by using the two distinct decision rules, while its specificities were 75.1% and 76.4%, respectively. In their study, the correlation between the BiliCam-predicted bilirubin level and the paired TSB level was 0.91. They concluded that BiliCam had an accuracy similar to that of TcB measurements, and it can provide near accurate predictions of TSB levels and could be used as a potential inexpensive screening tool to be effectively used in screening neonatal jaundice. However, one of the most influencing factors in the results of BiliCam is the race and ethnicity of the neonates (5).

Our smartphone-based estimation of bilirubin levels merely depends on a smartphone, a color calibration card, and a 100X zoom microscope. The use of the color calibration card helps to attenuate variations in the surrounding environment lighting conditions and facilitates image capture and data extraction. By utilizing the calibration card, all brands of Android smartphones with various lens qualities can be calibrated. All the processes in our smartphone-based bilirubin estimation are offline, so users can readily work with the application without any limitations.

Smartphone-based estimation of bilirubin level has the potential to be used for screening for neonatal jaundice, especially in areas with limited resources like low- and middle-income countries. It can be served as a low-cost technology to aid health care professionals and parents to easily screen for neonatal jaundice, in office settings and even during home visits, and reduce the morbidity and mortality of neonatal jaundice.