Application of adjusted-receiver operating characteristic curve analysis in combination of biomarkers for early detection of gestational diabetes mellitus

authors:

avatar Maedeh Amini , avatar Anoshirvan Kazemnejad ORCID , * , avatar Farid Zayeri , avatar Azam Amirian , avatar Nourossadat Kariman

Koomesh: Vol.21, issue 4; 751-758
published online: December 03, 2019
article type: Research Article
received: October 27, 2018
accepted: April 03, 2019

how to cite: Amini M, Kazemnejad A, Zayeri F, Amirian A, Kariman N. Application of adjusted-receiver operating characteristic curve analysis in combination of biomarkers for early detection of gestational diabetes mellitus. koomesh. 2019;21(4):e153141. 

Abstract

Introduction: In medical diagnostic field, evaluation of diagnostic accuracy of biomarkers or tests has always been a matter of concern. In some situations, one biomarker alone may not be sufficiently sensitive and specific for prediction of a disease. However, combining multiple biomarkers may lead to better diagnostic.  The aim of this study was to assess the performance of combination of biomarkers to early detection of gestational diabetes. Materials and Methods: In the present study, the information of 523 pregnant women referring to Mahdieh and Taleghani hospitals located in Tehran city was used. Relatively, Unconjugated Estriol (uE3), Alfa-Fetoprotein (AFP), and Beta- Human Chorionic Gonadotropin (β-HCG) were measured in the early second trimester of pregnancy. The accuracy of these biomarkers and also finding optimal linear combination of them was evaluated by area under receiver operating characteristic (ROC) curve (AUC), sensitivity, and specificity. Choosing of the best cut-point was based on Youden index. Data were statistically analyzed applying R 3.5.1 software package. Results: In the combination of two biomarkers, the largest AUC, sensitivity, and specificity values were 0.593, 61.90%, and 58.30%, respectively with optimal cut-point value 0.11. In the combination of the three biomarkers by adjusting for age and BMI effects simultaneously, the largest AUC, sensitivity, and specificity values were 0.751, 82.95%, and 74.62%, respectively with optimal cut-point value 0.10. Conclusion: Based on the findings of this research, the linear combination of the three biomarkers by considering covariate effects improved the diagnostic performance.  

References

  • 1.

    Moons KG, Biesheuvel CJ, Grobbee DE. Test research versus diagnostic research. Clin Chem 2004; 50: 473-476.

  • 2.

    Bossuyt PM, Irwig L, Craig J, Glasziou P. Diagnosis accuracy: assessing new tests against existing diagnostic pathways. BMJ 2006; 332: 1089-1092.

  • 3.

    Schnemann HJ, Oxman AD, Brozek J, Glasziou P, Jaeschke R, Vist GE, et al. Grading quality of evidence and strength of recommendations for diagnostic tests and strategies. BMJ 2008; 336: 1106-1110.

  • 4.

    Naaktgeboren CA, De Groot JA, van Smeden M, Moons KG, Reitsma JB. Evaluating diagnostic accuracy in the face of multiple reference standards. Ann Intern Med 2013; 159: 195-202.

  • 5.

    Linden A. Measuring diagnostic and predictive accuracy in disease management: an introduction to receiver operating characteristic (ROC) analysis. J Eval Clin Pract 2006; 12: 132-139.

  • 6.

    Cheam AS, McNicholas PD. Modelling receiver operating characteristic curves using Gaussian mixtures. Comput Stat Data Anal 2016; 93: 192-208.

  • 7.

    Haker S, Wells WM, Warfield SK, Talos IF, Bhagwat JG, Goldberg-Zimring D, et al. Combining classifiers using their receiver operating characteristics and maximum likelihood estimation. Med Image Comput Comput Assist Interv 2005; 8: 506-514.

  • 8.

    Leung Sf, Tam JS, Chan AT, Zee B, Chan LY, Huang DP, et al. Improved accuracy of detection of nasopharyngeal carcinoma by combined application of circulating EpsteinBarr virus DNA and anti-EpsteinBarr viral capsid antigen IgA antibody. Clin Chem 2004; 50: 339-345.

  • 9.

    Yuan Z, Ghosh D. Combining multiple biomarker models in logistic regression. Biometrics 2008; 64: 431-439.

  • 10.

    Ma S, Huang J. Combining multiple markers for classification using ROC. Biometrics 2007; 63: 751-757.

  • 11.

    Kang L, Liu A, Tian L. Linear combination methods to improve diagnostic/prognostic accuracy on future observations. Stat Methods Med Res 2016; 25: 1359-1380.

  • 12.

    Kwak JY, Hwang H, Kim S-K, Choi JY, Lee SM, Bang H, et al. Prediction of sarcopenia using a combination of multiple serum biomarkers. Sci Rep 2018; 8: 8574.

  • 13.

    Etzioni R, Kooperberg C, Pepe M, Smith R, Gann PH. Combining biomarkers to detect disease with application to prostate cancer. Biostatistics 2003; 4: 523-538.

  • 14.

    Guo S, Huang CC, Zhao W, Yang AC, Lin CP, Nichols T, Tsai SJ. Combining multi-modality data for searching biomarkers in schizophrenia. PloS One 2018; 13: e0191202.

  • 15.

    Huang X, Qin G, Fang Y. Optimal combinations of diagnostic tests based on AUC. Biometrics 2011; 67: 568-576.

  • 16.

    Su JQ, Liu JS. Linear combinations of multiple diagnostic markers. J Am Stat Assoc 1993; 88: 1350-1355.

  • 17.

    Liu C, Liu A, Halabi S. A minmax combination of biomarkers to improve diagnostic accuracy. Stat Med 2011; 30: 2005-2014.

  • 18.

    Kang L, Xiong C, Crane P, Tian L. Linear combinations of biomarkers to improve diagnostic accuracy with three ordinal diagnostic categories. Stat Med 2013; 32: 631-643.

  • 19.

    Unal I. Defining an optimal cut-point value in roc analysis: an alternative approach. Comput Math Methods Med 2017; 2017: 3762651.

  • 20.

    Li C, Tao H, Yang X, Zhang X, Liu Y, Tang Y, et al. Assessment of a combination of Serum Proteins as potential biomarkers to clinically predict Schizophrenia. Int J Med Sci 2018; 15: 900-906.

  • 21.

    Schisterman EF, Faraggi D, Reiser B. Adjusting the generalized ROC curve for covariates. Stat Med 2004; 23: 3319-3331.

  • 22.

    Janes H, Longton G, Pepe M. Accommodating covariates in ROC analysis. Stata J 2009; 9: 17-39.

  • 23.

    Liu D, Zhou XH. Covariate adjustment in estimating the area under ROC curve with partially missing gold standard. Biometrics 2013; 69: 91-100.

  • 24.

    Bagheri R, Hekmat K, Abedi P, Tabesh H, Omidifar Z. Relationship of gestational diabetes with serum levels of retinol. Koomesh 2014; 15: 551-556. (Persian).

  • 25.

    zkaya E, akr E, nar M, Altay M, Gelien O, Kara F. Second trimester serum alpha-fetoprotein level is a significant positive predictor for intrauterine growth restriction in pregnant women with hyperemesis gravidarum. J Turk Ger Gynecol Assoc 2011; 12: 220-224.

  • 26.

    Rty R, Anttila L, Virtanen A, Koskinen P, Laitinen P, Mrsky P, et al. Maternal midtrimester free HCG and AFP serum levels in spontaneous singleton pregnancies complicated by gestational diabetes mellitus, pregnancyinduced hypertension or obstetric cholestasis. Prenat Diagn 2003; 23: 1045-1048.

  • 27.

    Spandana T, Chaudhuri J, Silambanan S. Assessing the need for adjustment of first trimester screening markers in Diabetic Women. IJCBR 2015; 2: 190-193.

  • 28.

    Hur J, Cho EH, Baek KH, Lee KJ. Prediction of gestational diabetes mellitus by unconjugated estriol levels in maternal serum. Int J Med Sci 2017; 14: 123-127.

  • 29.

    Obuchowski NA, Bullen JA. Receiver operating characteristic (ROC) curves: review of methods with applications in diagnostic medicine. Phys Med Biol 2018; 63: 07TR01.

  • 30.

    Hajian-Tilaki K. Receiver operating characteristic (ROC) curve analysis for medical diagnostic test evaluation. Caspian J Intern Med 2013; 4: 627-635.

  • 31.

    Feng Z, Yasui Y. Statistical considerations in combining biomarkers for disease classification. Dis Markers 2004; 20: 45-51. ##.

  • 32.

    Erdman LK, Dhabangi A, Musoke C, Conroy AL, Hawkes M, Higgins S, et al. Combinations of host biomarkers predict mortality among Ugandan children with severe malaria: a retrospective case-control study. PloS One 2011; 6: e17440.

  • 33.

    Duan X, Yang Y, Tan S, Wang S, Feng X, Cui L, et al. Application of artificial neural network model combined with four biomarkers in auxiliary diagnosis of lung cancer. Med Biol Eng Comput 2017; 55: 1239-1248.

  • 34.

    Fong Y, Yin S, Huang Y. Combining biomarkers linearly and nonlinearly for classification using the area under the ROC curve. Stat Med 2016; 35: 3792-3809.

  • 35.

    Pepe MS, Cai T, Longton G. Combining predictors for classification using the area under the receiver operating characteristic curve. Biometrics 2006; 62: 221-229.

  • 36.

    Rydevik G, Innocent GT, Marion G, Davidson RS, White PC, Billinis C, et al. Using combined diagnostic test results to hindcast trends of infection from cross-sectional data. PLoS Comput Biol 2016; 12: e1004901.

  • 37.

    Wu K, Cheng Y, Li T, Ma Z, Liu J, Zhang Q, et al. The utility of HbA1c combined with haematocrit for early screening of gestational diabetes mellitus. Diabetol Metab Syndr 2018; 10: 14.

  • 38.

    Muhammad N, Coolen F, Coolen-Maturi T. Nonparametric predictive inference with parametric copulas for combining diagnostic tests. Stat Optim Inf Comput 2017; 6: 398-408.

  • 39.

    Yan L, Tian L, Liu S. Combining large number of weak biomarkers based on AUC. Stat Med 2015; 34: 3811-3830.

  • 40.

    Sancken U, Bartels I. Biochemical screening for chromosomal disorders and neural tube defects (NTD): is adjustment of maternal alphafetoprotein (AFP) still appropriate in insulindependent diabetes mellitus (IDDM)? Prenat Diagn 2001; 21: 383-386.

  • 41.

    Thornburg LL, Knight KM, Peterson CJ, McCall KB, Mooney RA, Pressman EK. Maternal serum alpha-fetoprotein values in type 1 and type 2 diabetic patients. Am J Obstet Gynecol 2008; 199: 135.

  • 42.

    Sayn NC, Canda MT, Ahmet N, Arda S, St N, Varol FG. The association of triple-marker test results with adverse pregnancy outcomes in low-risk pregnancies with healthy newborns. Arch Gynecol Obstet 2008; 277: 47-53.##.