Diagnosis of Breast Tumors with Sonographic Texture Analysis Using Run-length Matrix

authors:

avatar Ali Abbasian Ardakani 1 , avatar Afshin Mohammadi ORCID 2 , * , avatar Akbar Gharbali 3 , avatar Aram Rostami 1

Department of Medical Physics, Faculty of Medicine, Iran University of Medical Sciences, Tehran, Iran
Department of Radiology, Faculty of Medicine, Imam Khomeini Hospital, Urmia University of Medical Sciences, Urmia, Iran
Department of Medical Physics, Faculty of Medicine, Urmia University of Medical Sciences, Urmia, Iran

How To Cite Abbasian Ardakani A, Mohammadi A, Gharbali A, Rostami A. Diagnosis of Breast Tumors with Sonographic Texture Analysis Using Run-length Matrix. Int J Cancer Manag. 2018;11(2):e6120. https://doi.org/10.5812/ijcm.6120.

Abstract

Background:

Early detection and reliable diagnosis of breast cancer could lead to improved cure rates and reduce mortality and management costs.

Objectives:

To explore the potential of texture analysis based on run-length matrix features for classifying benign and malignant breast tumors in ultrasound imaging.

Methods:

A total of 70 breast tumors (38 benign and 32 malignant) have used in the proposed computer-aided diagnosis system. Twenty run-length matrix features have extracted for texture analysis in three normalizations (default, 3sigma, and 1% - 99%). Linear discriminant analysis and principal component analysis have employed to transform raw data to lower-dimensional spaces and increase discriminative power. The features have classified by the first nearest neighbor classifier.

Results:

The features under 3sigma normalization have designed via Linear discriminant analysis indicated high performance in classifying benign and malignant breast tumors with a sensitivity of 96.87%, specificity of 100%, accuracy of 98.57%, positive predictive value of 100%, and negative predictive value of 97.43%. The area under receiver operating characteristic curve was 0.992.

Conclusions:

Run-length matrix features had a high potential to characterize and could help radiologist to diagnosis breast tumors.

1. Background

Early detection of breast cancer could lead to improved cure rates and reduced mortality and management costs. An accurate and reliable diagnosis was the primary key to an early detection procedure. Although mammography was the “gold standard” modality for breast imaging, it had some limitations, such as missing over 10% of all cancers, especially in dense breasts (1). When ultrasound performed in conjunction with mammography, has reported to improve diagnostic accuracy (area under the receiver-operator characteristic (ROC) curve) from 0.78 to 0.91 (2). Also in 1995, Stavros et al. has reported that by using ultrasound modality, malignant tumors could be detected with sensitivity, specificity, and overall accuracy of 98.4%, 67.8%, and 72.9%, respectively (3). Real-time scanning, convenience, being radiation-free, and inexpensive were advantages making ultrasound examination a popular technique.

Biopsy was the best way to confirm whether a tumor would be benign or malignant, however, it was invasive, carried the risk of infection, and expensive, and only 10% - 31% of biopsied tumors were malignant (4, 5). To avoid unnecessary biopsy, assuage anxiety, and increased diagnostic confidence, computer-aided diagnosis (CAD) systems have been developed. CAD systems were tools using computer technology to identification and classification of breast masses.

Ultrasound image comprised diverse gray-level intensity, and different tissues had different texture. Although there was no precise and mathematical definition of texture, simply conceived by human eye. Image texture could be described by various patterns: coarse, fine, smooth, or spatial variations in pixel intensity of objects within an image. Structural abnormalities in ultrasound image could be extracted by visual inspection, but complex patterns were difficult to interpret. Computerized texture analysis was a mathematical technique that detected pathological changes that could not be perceived by the human eye using conventional ultrasound imaging (6-8).

2. Objectives

Many texture featured such as gray level co-occurrence matrix (GLCM) (9, 10), wavelet, shearlet and curvelet (10), contourlet (10, 11) and auto-covariance matrix (12) have proven to be useful in differentiation of benign and malignant breast tumors. This study has proposed a new CAD system based on run-length matrix (RLM) features to quantitatively characterize benign and malignant breast tumors. The variations of these features between benign and malignant tumors have used for classification. No previously published works that employed texture analysis with RLM features for ultrasound imaging have found.

3. Methods

3.1. Patients and Image Acquisition

Ultrasound images of 70 patients comprising 38 benign (33 fibroadenoma and 5 granulomatous mastitis) and 32 malignant breast tumors (28 invasive ductal carcinoma and 4 lobular carcinoma) with biopsy-proven have used in our retrospective study. All biopsies have performed after obtaining ultrasound images. Ultrasound images have performed using Accuvix V20 sonography system (Medison, Seoul, Korea) equipped with L5-13IS (5 - 13 MHz) linear array transducer. The images have collected prospectively from January 11 to June 31, 2015.

3.2. Texture Analysis Based on Run-length Matrix

MaZda software version 4.6 (7) (The technical University of Lodz, institute of electronics) have used for texture analysis and classification. One 2D region of interest (ROI) has selected for each tumor; hence 70 non-overlap ROIs (38 benign, and 32 malignant) have selected for discrimination and classification. To avoid bias in the selection of ROI for texture analysis, ROI have defined on images by a single experienced radiologist. The ROIs size has depended on tumors, where the mean size of ROI was 15 × 15 mm. General surgeon has referred the cases based on clinical indication. The breast tumors images have compared with biopsy and the reference for statistical analysis. Since this study has based on texture analysis and our gold standard was pathology, other tumor characteristic features do not have any additional information and not considered.

This study has proposed texture analysis using RLM features to distinguish benign and malignant breast tumors (13). The RLM p(i, j) has searched the image in a given direction for runs of pixels having the same grey-level value. The matrix element (i, j) has represented the number of times the image contained pixels having gray level i and run of length j in the given direction. The current study has focused on five numerical texture measures obtained from the RLM: short run emphasis (ShrtREmph), long run emphasis (LngREmph), gray level nonuniformity (GLevNonUni), run length nonuniformity (RLNonUni), and fraction of the image in runs (Fraction). They have defined as:

Equation 1:
Equation 2:
Equation 3:
Equation 4:
Equation 5:

where Na and Nr were the number of gray levels in the image and the number of different run lengths that occur, respectively. These five parameters have calculated in four directions: horizontal, vertical, 45-degree, and 135-degree. In MaZda software, the names of RLM features contained an abbreviation that defines angle and feature type; for example, 135dgr- GLevNonUni means the gray level nonuniformity has calculated in a 135-degree direction.

3.3. Pixel Normalization

In this study, 20 RLM features have calculated for each ROI in three normalizations. The first, N1, was the default, and no normalization has applied. Thus, images had the same appearance at an intensity range of 1 to 2k, where k was the number of bits per pixel. The second, N2, was 3-sigma, in which the ROI intensities have limited in the range (µ-3σ and µ+3σ) where µ and σ, respectively, were the mean value and standard deviation of the intensity. The third normalization was N3, 1% - 99%, in which the image intensity ranges were normalized between the darkness level (accumulated histogram is equal to 1% of total) and the brightness level (accumulated histogram = 99% of total) inside the ROI. The intensity levels outside the normalization range have not considered for analysis.

3.4. Statistical Analysis and Classification

To transform raw data to lower-dimensional spaces and increase the discriminative power, linear discriminant analysis (LDA) and principal component analysis (PCA) have used (14, 15). First nearest neighbor (1-NN) classifier has used for the features resulting from LDA and PCA (16). In order to compare the performance of diagnostic, five well-known indexes have calculated: accuracy (ACC), sensitivity (SEN), specificity (SPC), positive predictive value (PPV), negative predictive value (NPV). Their definitions have given as:

Equation 6:
Equation 7:
Equation 8:
Equation 9:
Equation 10:

Where NTP and NTN were the number of correctly diagnosed malignant and benign cases respectively. NFP and NFN were the number of incorrectly diagnosed malignant and benign cases respectively.

Area under the ROC curve (Az) has also calculated to evaluate the overall performance of proposed method (17). ROC analysis has performed with the SPSS software (SPSS Inc., Chicago, USA) and the Az values have estimated beyond the 95% confidence level. Figure 1 has shown the steps of CAD processing.

Overview of General Texture Analysis Process in the Ultrasound Breast Image
Overview of General Texture Analysis Process in the Ultrasound Breast Image

4. Results

Briefly in this study, six options for texture analysis have utilized: three normalization schemes and two texture analysis methods. A total of 70 pathology-proven cases with 38 benign and 32 malignant breast tumors have selected for the evaluation of classification accuracy using the proposed method.

The diagnostic performances of the texture analysis methods and normalization schemes have listed in Table 1. Considering the normalization role, in default, RLM features have represented a higher performance with 87.5% sensitivity, 92.1% specificity, 90% accuracy, 90.32% PPV, and 89.74% NPV by LDA in comparison with PCA, while classification tasks carried out by PCA yielded a lower discrimination performance in which sensitivity, specificity, accuracy, PPV, and NPV were 84.37%, 86.84%, 85.71%, 84.37%, and 86.84%, respectively.

Table 1.

Diagnostic Performance of Proposed Method in Three Normalization Schemesa

NormalizationMethod of Feature ReductionSENSPCACCPPVNPVAz Valueb
DefaultLDA87.592.109090.3289.740.963 (0.923, 1.000)
PCA84.3786.8485.7184.3786.840.939 (0.886, 0.992)
3sigmaLDA96.8710098.5710097.430.992 (0.977, 1.000)
PCA87.592.109090.3289.740.947 (0.898, 0.995)
1% - 99%LDA93.7510097.14100950.979 (0.953, 1.000)
PCA90.6297.3694.2896.6692.50.949 (0.902, 0.996)

In 3-sigma normalization the LDA has shown better discrimination power with a sensitivity of 96.87%, specificity of 100%, accuracy of 98.57%, PPV of 100%, and NPV of 97.43%. The features analyzed by PCA has shown lower discrimination power than LDA with sensitivity, specificity, accuracy, PPV, and NPV measuring 87.5%, 92.10%, 90%, 90.32%, and 89.74%, respectively.

In 1% - 99% normalization features analyzed by LDA had a higher discrimination power than those by PCA to distinguish between benign and malignant breast tumors, in which sensitivity, specificity, accuracy, PPV, and NPV were 93.75%, 100%, 97.14%, 100%, and 95%, respectively. PCA has indicated a lower performance in which sensitivity, specificity, accuracy, PPV, and NPV were 90.62%, 97.36%, 94.28%, 96.66%, and 92.5%, respectively.

The discriminatory power of the RLM features extracted has examined by constructing ROC curves. Figure 2 has represented the ROC curves of LDA and PCA for the proposed CAD system. They have plotted on the same graph for each normalization to compare the discriminating power for classification of benign and malignant breast tumors. In general, the LDA method had an advantage over PCA in each normalization with respect to a greater Az value. As shown, in the case of LDA, the best performance gains in 3-sigma normalization with an Az value of 0.992 which corresponded to a sensitivity of 96.87%, specificity of 100%, accuracy of 98.57%, PPV of 100%, and NPV of 97.43%. PCA has also shown the best accuracy in 1-99% normalization with an Az value of 0.949 which has corresponded to a sensitivity of 90.62%, specificity of 97.36%, accuracy of 94.28%, PPV of 96.66%, and NPV of 92.5%.

The Diagrams of the ROC Curve for Texture Analysis Method in Default A, in 3sigma B and in 1% - 99% Normalizations C
The Diagrams of the ROC Curve for Texture Analysis Method in Default A, in 3sigma B and in 1% - 99% Normalizations C

The data distributions of two texture classes of the best results in PCA and LDA have illustrated in Figure 3. As seen, LDA had the greatest power of discrimination to distinguish between benign and malignant breast tumors.

Sample Distributions After Two Texture Analysis Methods. A PCA; B LDA. MEF: Most Expressive Features; MDF: Most Discriminating Features; “1” and “2” Represented Benign and Malignant Breast Tumors Respectively
Sample Distributions After Two Texture Analysis Methods. A PCA; B LDA. MEF: Most Expressive Features; MDF: Most Discriminating Features; “1” and “2” Represented Benign and Malignant Breast Tumors Respectively

5. Discussion

Discrimination between benign and malignant tumors was one of the most critical factors to improving the initial diagnosis accuracy of radiologists. The results of the current study has proved that RLM texture features could differentiate between benign and malignant breast tumors with high accuracy. The results have demonstrated that LDA had more discriminative power than PCA in the data of this research. In all normalization schemes in order to Az values, LDA had a higher performance, as seen in Figure 2, whereas in 3-sigma, LDA had a much higher accuracy than PCA (0.992 vs. 0.947).

From Table 1 one could see that normalization reflected improvement on the performance of the classifier. Therefore, the best performance in this study has obtained in 3-sigma normalization with LDA. Moreover, PCA had the best performance in 1% - 99% normalization.

In the last decade, computer-aided diagnosis has employed to classify breast tumors using ultrasound imaging. Ultrasound imaging has included many kind of features such a textural, elastographic, and morphological features which were useful for classification tasks (18).

Elastographic features contain information about tissue stiffness. Ultrasound elastography was a non-invasive technique used to achieve tissue deformation in response to compression (19). Studies have used ultrasound elastography to classify benign and malignant breast tumors. Xiao et al. (20) used supersonic shear wave imaging elastography to extract elastographic features and classify benign and malignant breast tumors on ultrasound images. They have achieved a sensitivity of 90.9%, specificity of 97.5%, accuracy of 95.2%, PPV of 95.2%, NPV of 95.2%, and Az of 0.97 using a support vector machine classifier. Zhang et al. (21) used real-time ultrasound elastography to distinguish benign from malignant breast tumors with a sensitivity of 92.5%, specificity of 94.9%, accuracy of 93.8%, PPV of 93.9%, NPV of 93.7%, and Az of 0.96.

Morphological features has described margin irregularity, symmetry and shape of the tumor surface. In this regard, Chen et al. (22) has used a fractal feature and K-means classifier to classify benign and malignant breast tumors achieving a sensitivity of 93.64%, specificity of 84.29%, accuracy of 88.8%, PPV of 82.4%, NPV of 94.4%, and Az of 0.9218. Moon et al. (9) Moon has shown that combining shape and ellipsoid fitting features achieved a performance in which sensitivity, specificity, accuracy, PPV, NPV, and Az were 84.51%, 85.53%, 85.03%, 84.51%, 85.53%, and 0.9466, respectively. Wu et al. (12) has extracted morphological features from ultrasound images. They have used a support vector machine to classify breast tumors with a sensitivity of 88.89%, specificity of 92.5%, accuracy of 90.95%, PPV of 89.89%, NPV of 91.47%, and Az of 0.9389.

In some studies, features from different groups have combined to reach the best performance. Wu et al. (12) has combined texture and morphological features from ultrasound images and indicated that combined features gained better for classifying breast tumors with a sensitivity, specificity and accuracy of 96.67%, PPV of 95.6%, NPV of 97.48%, and Az of 0.9827. In a Moon et al. study (9), however, the combination of GLCM shape and ellipsoid fitting features had a disruptive effect on performance. The best results hare driven by morphological features with a sensitivity of 84.51% vs. 83.1% (morphological vs. combined features), specificity of 85.53% vs. 81.58%, accuracy of 85.03% vs. 82.31%, PPV of 84.51% vs. 80.82%, NPV of 85.53% vs. 83.78%, and Az of 0.9466 vs. 0.9388.

Another technique which could be used to differentiate benign from malignant breast tumors is contrast-enhanced sonography. Liu et al. (23) has indicated that contrast-enhanced sonography was useful for classifying breast tumors. They were able to classify benign and malignant breast tumors with a sensitivity of 72.7%, specificity of 80%, and accuracy of 78.2%.

In this work, 20 RLM features have used and presented good classification with a sensitivity of 96.87%, specificity of 100%, accuracy of 98.57%, PPV of 100%, and NPV of 97.43%. The area under receiver operating characteristic curve was 0.992. It has meant that about 99% of patients with uncertain malignant or benign breast cancer could avoid further examination if our texture analysis method has used for the diagnosis of these patients. In this study, three normalizations and two texture data analysis methods provide all together 6 states per ROI. Each set examined individually to find out the best features descriptor for differentiation benign and malignant breast tumor. Based on our hypothesis such condition (under LDA and 3sigma normalization) would classify groups with more confidence as it has shown in results. The current research has indicated that RLM features perform higher than other texture subsets employed in other studies, i.e. GLCM (9, 10), wavelet, shearlet and curvelet (10), contourlet (10), auto-covariance matrix (12) and a combination of texture features (10, 24). Results of the current study have also indicated that RLM texture analysis possesses significantly more discriminative ability than other methods, such as morphology and elastography (9, 12, 20-22). Likewise, the proposed method has demonstrated a more reliable performance in comparison with previous studies that combined texture and morphological features (9, 12).

Some studies have employed three-dimensional (3D) breast ultrasound image to distinguish between benign and malignant breast masses (9, 25, 26). In this regard, Chen et al. (25) has extracted shape, margin, lesion boundary, echogenicity, posterior acoustic features, and surrounding tissue and was able to classify breast tumors with a sensitivity, specificity, accuracy, PPV, and NPV of 89.6%, 92.8%, 95.9%, 84.5%, and 95.3%, respectively. Tan el al. (26) has used automated 3D breast ultrasound and reported an Az of 0.92 in discriminating benign from malignant breast lesions. The presented study, however, has used texture features from 2D ultrasound images, and its results have indicated that 2D ultrasound images were more favorable for the study of breast cancer. However, 3D ultrasound imaging has not conventionally used in most clinical settings.

There were several limitations in the current study. First, the data group was small, Further investigation using a larger data set would be needed. Second, in MaZda software feature combination tools were not available. This made it difficult to perform some calculations. For example, averaging of Run-length matrix features of four different orientations was difficult. Third, the radiologist’s diagnostic has not implemented in this study. The texture analysis results have compared only with pathology. Further investigation comparing the texture analysis results with radiologist diagnostic to evaluate radiologists’ performance would be needed.

This method was not an alternative for biopsy, but it could help radiologists select patients with a high malignancy risk for breast tumors for biopsy. The methods has used in this study could aid radiologists in distinguishing between benign and malignant breast tumors in the target tumors. The main advantage of this method was this fact that there would be no operator dependency, because analysis has performed by the computer. Moreover, it has required no additional time or costs.

MaZda software has developed in 1998 for the purpose of texture analysis of magnetic resonance imaging (6). We have used it also for ultrasound images which were useful for texture pattern recognition. Generally, our results has indicated that texture analysis was useful tool for discrimination of breast cancers by ultrasound images and could be an auxiliary tool to help radiologists improving accuracy regarding the classification of benign and malignant breast tumors.

In conclusion, we have proposed a new approach that has based on RLM features to differentiate benign and malignant tumors on ultrasound image. The experiment results have shown that RLM features had a potential to characterize and classify breast tumors.

5.1. Implication of the Manuscript

- Computerized texture analysis (CTA) was an accurate noninvasive method for assessing breast tumors.

- CTA based on run-length matrix (RLM) features has employed for the first time to differentiate between benign and malignant breast tumors in ultrasound images.

- RLM features have demonstrated a more reliable performance in comparison with other texture features groups and other methods, such as morphology, elastography and contrast-enhanced sonography.

- RLM features could help radiologists select patients with a high malignancy risk for breast tumors for biopsy.

- RLM features were useful for pattern recognition and have a potential to classify breast tumors.

Acknowledgements

References

  • 1.

    Jackson VP, Hendrick RE, Feig SA, Kopans DB. Imaging of the radiographically dense breast. Radiology. 1993;188(2):297-301. [PubMed ID: 8327668]. https://doi.org/10.1148/radiology.188.2.8327668.

  • 2.

    Berg WA, Blume JD, Cormack JB, Mendelson EB, Lehrer D, Bohm-Velez M, et al. Combined screening with ultrasound and mammography vs mammography alone in women at elevated risk of breast cancer. JAMA. 2008;299(18):2151-63. [PubMed ID: 18477782]. https://doi.org/10.1001/jama.299.18.2151.

  • 3.

    Stavros AT, Thickman D, Rapp CL, Dennis MA, Parker SH, Sisney GA. Solid breast nodules: use of sonography to distinguish between benign and malignant lesions. Radiology. 1995;196(1):123-34. [PubMed ID: 7784555]. https://doi.org/10.1148/radiology.196.1.7784555.

  • 4.

    Rubin M, Horiuchi K, Joy N, Haun W, Read R, Ratzer E, et al. Use of fine needle aspiration for solid breast lesions is accurate and cost-effective. Am J Surg. 1997;174(6):694-6. discussion 697-8. [PubMed ID: 9409599]. https://doi.org/10.1016/S0002-9610(97)00192-X.

  • 5.

    Meyer JE, Eberlein TJ, Stomper PC, Sonnenfeld MR. Biopsy of occult breast lesions. Analysis of 1261 abnormalities. JAMA. 1990;263(17):2341-3. [PubMed ID: 2157903]. https://doi.org/10.1001/jama.1990.03440170063037.

  • 6.

    Materka A. Texture analysis methodologies for magnetic resonance imaging. Dialogues Clin Neurosci. 2004;6(2):243-50. [PubMed ID: 22033841].

  • 7.

    Szczypinski PM, Strzelecki M, Materka A, Klepaczko A. MaZda--a software package for image texture analysis. Comput Methods Programs Biomed. 2009;94(1):66-76. [PubMed ID: 18922598]. https://doi.org/10.1016/j.cmpb.2008.08.005.

  • 8.

    Castellano G, Bonilha L, Li LM, Cendes F. Texture analysis of medical images. Clin Radiol. 2004;59(12):1061-9. [PubMed ID: 15556588]. https://doi.org/10.1016/j.crad.2004.07.008.

  • 9.

    Moon WK, Shen YW, Huang CS, Chiang LR, Chang RF. Computer-aided diagnosis for the classification of breast masses in automated whole breast ultrasound images. Ultrasound Med Biol. 2011;37(4):539-48. [PubMed ID: 21420580]. https://doi.org/10.1016/j.ultrasmedbio.2011.01.006.

  • 10.

    Zhou S, Shi J, Zhu J, Cai Y, Wang R. Shearlet-based texture feature extraction for classification of breast tumor in ultrasound image. Biomed Signal Process Control. 2013;8(6):688-96. https://doi.org/10.1016/j.bspc.2013.06.011.

  • 11.

    Zhang Q, Xiao Y, Chen S, Wang C, Zheng H. Quantification of elastic heterogeneity using contourlet-based texture analysis in shear-wave elastography for breast tumor classification. Ultrasound Med Biol. 2015;41(2):588-600. [PubMed ID: 25444693]. https://doi.org/10.1016/j.ultrasmedbio.2014.09.003.

  • 12.

    Wu WJ, Lin SW, Moon WK. An Artificial Immune System-Based Support Vector Machine Approach for Classifying Ultrasound Breast Tumor Images. J Digit Imaging. 2015;28(5):576-85. [PubMed ID: 25561066]. https://doi.org/10.1007/s10278-014-9757-1.

  • 13.

    Galloway MM. Texture analysis using gray level run lengths. Comput Grf Image Proc. 1975;4(2):172-9. https://doi.org/10.1016/s0146-664x(75)80008-6.

  • 14.

    Webb AR. Introduction to Statistical Pattern Recognition. Statistical Pattern Recognition. John Wiley & Sons, Ltd; 2003. p. 1-31.

  • 15.

    Fukunaga K. Introduction to statistical pattern recognition. Academic press; 2013.

  • 16.

    Anderson J, Pellionisz A, Rosenfeld E. Neurocomputing II. Directions for Research. MIT Press. Cambridge, MA; 1990.

  • 17.

    van Erkel AR, Pattynama PM. Receiver operating characteristic (ROC) analysis: basic principles and applications in radiology. Eur J Radiol. 1998;27(2):88-94. [PubMed ID: 9639133]. https://doi.org/10.1016/S0720-048X(97)00157-5.

  • 18.

    Jalalian A, Mashohor SB, Mahmud HR, Saripan MI, Ramli AR, Karasfi B. Computer-aided detection/diagnosis of breast cancer in mammography and ultrasound: a review. Clin Imaging. 2013;37(3):420-6. [PubMed ID: 23153689]. https://doi.org/10.1016/j.clinimag.2012.09.024.

  • 19.

    Ophir J, Cespedes I, Ponnekanti H, Yazdi Y, Li X. Elastography: a quantitative method for imaging the elasticity of biological tissues. Ultrason Imaging. 1991;13(2):111-34. [PubMed ID: 1858217]. https://doi.org/10.1177/016173469101300201.

  • 20.

    Xiao Y, Zeng J, Niu L, Zeng Q, Wu T, Wang C, et al. Computer-aided diagnosis based on quantitative elastographic features with supersonic shear wave imaging. Ultrasound Med Biol. 2014;40(2):275-86. [PubMed ID: 24268454]. https://doi.org/10.1016/j.ultrasmedbio.2013.09.032.

  • 21.

    Zhang X, Xiao Y, Zeng J, Qiu W, Qian M, Wang C, et al. Computer-assisted assessment of ultrasound real-time elastography: initial experience in 145 breast lesions. Eur J Radiol. 2014;83(1):e1-7. [PubMed ID: 24148563]. https://doi.org/10.1016/j.ejrad.2013.09.009.

  • 22.

    Chen DR, Chang RF, Chen CJ, Ho MF, Kuo SJ, Chen ST, et al. Classification of breast ultrasound images using fractal feature. Clin Imaging. 2005;29(4):235-45. [PubMed ID: 15967313]. https://doi.org/10.1016/j.clinimag.2004.11.024.

  • 23.

    Liu H, Jiang Y, Dai Q, Zhu Q, Wang L, Zhang J, et al. Differentiation of benign and malignant sub-1-cm breast lesions using contrast-enhanced sonography. J Ultrasound Med. 2015;34(1):117-23. [PubMed ID: 25542947]. https://doi.org/10.7863/ultra.34.1.117.

  • 24.

    Chen DR, Huang YL, Lin SH. Computer-aided diagnosis with textural features for breast lesions in sonograms. Comput Med Imaging Graph. 2011;35(3):220-6. [PubMed ID: 21131178]. https://doi.org/10.1016/j.compmedimag.2010.11.003.

  • 25.

    Chen L, Chen Y, Diao XH, Fang L, Pang Y, Cheng AQ, et al. Comparative study of automated breast 3-D ultrasound and handheld B-mode ultrasound for differentiation of benign and malignant breast masses. Ultrasound Med Biol. 2013;39(10):1735-42. [PubMed ID: 23849390]. https://doi.org/10.1016/j.ultrasmedbio.2013.04.003.

  • 26.

    Tan T, Platel B, Twellmann T, van Schie G, Mus R, Grivegnee A, et al. Evaluation of the effect of computer-aided classification of benign and malignant lesions on reader performance in automated three-dimensional breast ultrasound. Acad Radiol. 2013;20(11):1381-8. [PubMed ID: 24119350]. https://doi.org/10.1016/j.acra.2013.07.013.