Skip to main content

Review of medical image recognition technologies to detect melanomas using neural networks

Abstract

Background

Melanoma is one of the most aggressive types of cancer that has become a world-class problem. According to the World Health Organization estimates, 132,000 cases of the disease and 66,000 deaths from malignant melanoma and other forms of skin cancer are reported annually worldwide (https://apps.who.int/gho/data/?theme=main) and those numbers continue to grow. In our opinion, due to the increasing incidence of the disease, it is necessary to find new, easy to use and sensitive methods for the early diagnosis of melanoma in a large number of people around the world. Over the last decade, neural networks show highly sensitive, specific, and accurate results.

Objective

This study presents a review of PubMed papers including requests «melanoma neural network» and «melanoma neural network dermatoscopy». We review recent researches and discuss their opportunities acceptable in clinical practice.

Methods

We searched the PubMed database for systematic reviews and original research papers on the requests «melanoma neural network» and «melanoma neural network dermatoscopy» published in English. Only papers that reported results, progress and outcomes are included in this review.

Results

We found 11 papers that match our requests that observed convolutional and deep-learning neural networks combined with fuzzy clustering or World Cup Optimization algorithms in analyzing dermatoscopic images. All of them require an ABCD (asymmetry, border, color, and differential structures) algorithm and its derivates (in combination with ABCD algorithm or separately). Also, they require a large dataset of dermatoscopic images and optimized estimation parameters to provide high specificity, accuracy and sensitivity.

Conclusions

According to the analyzed papers, neural networks show higher specificity, accuracy and sensitivity than dermatologists. Neural networks are able to evaluate features that might be unavailable to the naked human eye. Despite that, we need more datasets to confirm those statements. Nowadays machine learning becomes a helpful tool in early diagnosing skin diseases, especially melanoma.

Introduction

Melanoma is one of the most malignant and rapidly progressing neoplasms registered in humans. The American Cancer Society estimated that in 2019, about 96,480 new cases of melanoma would be detected in the United States, and the death rate will be 7230 [1]. The statistics of the Ministry of Health of the Russian Federation showed the incidence rate of melanoma in 2015 for 6.99 per 100 thousand people: the incidence among women is higher (7.97) than among men (5.86) [2].

Background

Despite the great significance of the problem, there is still no substantial etiological factor for melanoma. Established that the risk of malignant melanocytes is increased by the ultraviolet radiation of a certain wavelength of type A and B (290–400 nm) [3]. On the one hand, it is established that researchers detected rare mutations, often inherited, which significantly increased the risk of melanoma. One class of mutations affects the gene CDKN2A (cyclin-dependent kinase inhibitor 2A) [4], which leads to the destabilization of p53, a transcription factor involved in apoptosis. Loss of the function of CDK4 (cyclin-dependent kinase 4) [5], a kinase involved in cell differentiation, is also associated with this gene. It is proved that people with mutations in the MCR1 gene responsible for red hair color have a risk of developing melanoma twice as high as the rest of the population [6, 7]. Familial cases of melanoma (FAMMM) are also associated with CDKN2A mutations, but this factor has low specificity.

Habitat plays a role: families living in areas with a high degree of insolation have a greater risk of melanoma occurrence; especially if all family members have light skin color (people with fair skin have a 20-fold higher risk of developing melanoma than representatives of the Negroid race) [7,8,9].

Diagnosing melanoma

Visual inspection is the most common method for diagnosing skin melanoma. The most common is the ABCDE algorithm where A is «asymmetry», B is «borders», C is «color», D is «diameter», E is growing with time (evolving) [10, 11]. Some doctors prefer using E as an elevation above the skin. The combination of these signs has high sensitivity and specificity. A dermatoscopic method with such an algorithm allows detecting malignancy in early stages, especially in patients in risk groups for the development of melanoma. Patient education plays a significant role, including an explanation of the value and methods of periodic self-examination [11, 12].

The usage or the application of neural networks in melanoma diagnosing

The problem of malignant skin diseases was relevant for people in any era. The first scientific literature on the treatment of melanoma appeared in the eighteenth century [13]. Rene Laennec first described melanoma as a disease [14]. General practitioner William Norris was the first to report on melanoma in English from Stourbridge, England, in 1820 [15]. The first official recognition of the progressive melanoma as non-treatable came from Samuel Cooper in 1840 [16]. He stated the only chance for a cure depends on the early treatment of the disease (that is, the early removal of a malignant mole).

In 1991, Cohen and Hudson published the work «Neural network approach to detection of metastatic melanoma from a chromatographic analysis of urine» [17]. The authors describe a neural net analysis of comparative data on urine chromatography of three groups of patients. Due to those findings, scientists noted great potential in the use of neural networks in the diagnosis of melanoma, having received the correct diagnosis from the neural network in more than 80% of cases. Afterward, neural networks for the diagnosis of melanoma were trained mainly for the analysis of dermatoscopic images. In 1994, scientists published a paper in which the neural network already conducts an analysis of color images. Their method was based on the shape and color of the tumor, which was analyzed by the neural network to classify the formations in images as malignant or benign. In their study, the authors received more than 80% of the correct diagnosis on real images of skin tumors [18].

Objective

This study presents a review of PubMed papers including requests «melanoma neural network» and «melanoma neural network dermatoscopy». We review recent researches and discuss their opportunities acceptable in clinical practice.

Methods

Search strategy

For this review, we searched through the PubMed database. According to PubMed, the first publications for the request «melanoma neural network» date back to 1991. Since then, the number of publications has grown unevenly every year, and the peak was in 2005 (12 publications) and 2018 (22 publications). As of May 2019, there are 138 publications on this request. At the request «melanoma neural network dermatoscopy» PubMed provides a list of 40 publications most of which relate to 2016–2019 indicating a growing interest in the early diagnosis of melanoma by analyzing dermatoscopy images using machine learning. Only papers that reported results, progress and outcomes are included in this review. In total, we found 157 unique papers, and 11 papers were matching our criteria.

Study selection

We have limited our review to articles that, in our opinion, are the most significant for assessing the diagnostic opportunities of neural networks. Only papers that showed results, progress and outcomes are included in this review. This latter criterion includes presenting the approaches in an understandable manner and discussing the results sufficiently as well as the process of neural network implementation resulted in overall progress.

Results

Neural networks nowadays

By 2019, the accuracy of diagnostics using the ABCD algorithm and its additions has increased and, according to some data, has begun to exceed the results of dermatologists. In a study published in March 2019, the neural network had a smaller percentage of erroneous results compared to dermatologists (145 people), which indicates higher reliability of computer vision compared to human assessment for the tasks of classifying dermatological images [19].

German scientists published a resonant study in 2016, which proposed a comparison to 58 dermatologists [20]. Discussion of this research in the scientific community has led many to conclude that convoluted neural networks can serve as an excellent diagnostic aid possessing greater sensitivity and specificity which will affect the cost of treatment, in particular, reducing the number of unnecessary operations. Regardless of the experience of any physician, we can benefit from helping to classify images with a neural network.

At a certain point, researchers faced the problem of image quality and the amount of noise on them. Researchers using FC (fuzzy clustering) models took up this. Clustering was included in the study with a convoluted neural network, which significantly increased the quality of analysis of dermatoscopic images.

In a recent study, researchers suggested that errors in diagnosing malignancy of melanomas could be associated with damage to images and the criterion of the boundaries of melanoma [21]. They decided to include in the backpropagation (BP) neural network an algorithm to improve the image quality and modify the boundary criterion, ultimately having the result identical in specificity but exceeding other methods in sensitivity and accuracy. They concluded that previously proposed characteristics of the boundaries of the formations were too sensitive to noise in the images, which led to a decrease in the sensitivity and accuracy.

Scientists studying FC models in convolutional neural networks (CNN) in their study concluded that using an effective training mechanism, a neural network can detect signs of various skin diseases of the same patient, as well as various diseases of numerous patients. This makes it possible to use neural networks in the diagnosis of a wider range of skin diseases [22].

In 2017, Nature published a study in which researchers trained neural networks to distinguish keratinocyte carcinomas from benign seborrheic keratosis and malignant melanomas from benign nevi. They said that the fast, scalable method could be on mobile devices and had the potential for significant clinical impact, including it in primary care and improving clinical decision-making for specialists in dermatology [23].

In 2018, a hybrid of the World Cup Optimization (WCO) algorithm and an artificial neural network was proposed, which made it possible to avoid falling into a local minimum and increased the speed of analysis. This approach has expanded the capabilities of the neural network and increased the frequency of making correct diagnoses [24]. In the same year, a multiparameter neural network algorithm was proposed. At the input level, this algorithm included patient history data such as gender, age, disease history (bronchial asthma, diabetes mellitus, cardiovascular diseases), ethnicity, etc. This algorithm is cheap, non-invasive, easy to use, and could become available for patients to calculate the risk without the direct participation of the doctor. The researchers plan to expand the list of factors, including the cases of ultraviolet irradiation [25]. In our opinion, the inclusion of additional parameters, such as data from genetic research and family history, can significantly improve the output of the neural network.

All those studies are based on the fact that the most important factor for a good prognosis for melanoma is early diagnosis. In 2018 Indian researchers trained the neural network to analyze images from “handheld imaging devices” instead of from stationary dermatoscopic devices [26]. Such more accessible devices give more prospects for early correct diagnosis. In general, we can conclude that for the macro images, the proposed model provides optimal efficiency in the diagnosis of melanoma.

In addition, German researchers performed a new study in 2019. Results showed that a convolutional neural network trained by open-source images outperformed 136 of the 157 dermatologists and all the different levels of experience [27].

Discussion

The analysis of this study leads us to the following statements:

  • Neural networks in 2019 have greater sensitivity and specificity than dermatologists

  • A neural network can evaluate features that might be unavailable to the naked human eye

  • CNN provides the possibility of early detection of signs of melanoma and early treatment

  • The need for staff training and the purchase of expensive equipment for creating dermatoscopic images can be replaced by software using FC-neural networks

By 2019, neural networks show significantly higher sensitivity, specificity and accuracy in comparison with the previous versions and work of dermatologists. The prospect of using this method as an auxiliary and later, perhaps, the main diagnostic is expressed in its increasing availability, accuracy and ease of use.

Conclusions

In the age of rapid technology development, information systems and machine learning, the neural network is a learning algorithm that can greatly facilitate and improve the level of accuracy and timeliness of diagnostics.

Availability of data and materials

All papers are available on publisher websites. All data generated or analyzed during this study are included in this published article.

Abbreviations

BP:

Backpropagation

CNN:

Convolutional neural networks

FAMMM:

Familial Atypical Multiple Mole Melanoma Syndrome

FC:

Fuzzy clustering

WCO:

World Cup Optimization

ABCD algorithm:

Algorithm based on asymmetry, border, color, and differential structures

References

  1. 1.

    Marc Hurlbert. MRA Chief Science Officer: Estimated Rates of Melanoma Increasing in 2019. The Melanoma Research Alliance 9 January 2019.

  2. 2.

    Kabanova MA. The Epidemiological Situation of Skin Melanoma in Russia in the Year 2015. Sci J «Current problems of health care and medical statistics». 2017;3:23–38.

    Google Scholar 

  3. 3.

    Boniol M, Autier P, Boyle P, Gandini S. Cutaneous melanoma attributable to sunbed use: systematic review and meta-analysis. BMJ (Clinical Research Ed.). 2012;345:e4757. https://doi.org/10.1136/bmj.e4757 PMC 3404185. PMID 22833605.

    Article  Google Scholar 

  4. 4.

    Goydos JS, Shoen SL. Acral Lentiginous Melanoma. Cancer Treat Res. 2016;167:321–9. https://doi.org/10.1007/978-3-319-22539-5_14 ISBN 978-3-319-22538-8. PMID 26601870.

    Article  PubMed  Google Scholar 

  5. 5.

    Helgadottir H, Höiom V, Tuominen R, Jönsson G, Månsson-Brahme E, Olsson H, Hansson J. CDKN2a mutation-negative melanoma families have increased risk exclusively for skin cancers but not for other malignancies. Int J Cancer. 2015;137(9):2220–6. https://doi.org/10.1002/ijc.29595 Epub 2015 May 21.

    Article  PubMed  CAS  Google Scholar 

  6. 6.

    Feller L, Khammissa RA, Kramer B, Altini M, Lemmer J. Basal cell carcinoma, squamous cell carcinoma and melanoma of the head and face. Head Face Med. 2016;12:11. https://doi.org/10.1186/s13005-016-0106-0. PMC 4744388. PMID 26850723.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  7. 7.

    Bliss J, Ford D, Swerdlow A, Armstrong B, Cristofolini M, Elwood J, Green A, Holly E, Mack T, MacKie R. Risk of cutaneous melanoma associated with pigmentation characteristics and freckling: systematic overview of 10 case-control studies. The international melanoma analysis group (IMAGE). Int J Cancer. 1995;62(4):367–76. https://doi.org/10.1002/ijc.2910620402 PMID 7635560.

    Article  PubMed  CAS  Google Scholar 

  8. 8.

    Wang S, Setlow R, Berwick M, Polsky D, Marghoob A, Kopf A, Bart R. Ultraviolet A and melanoma: a review. J Am Acad Dermatol. 2001;44(5):837–46. https://doi.org/10.1067/mjd.2001.114594 PMID 11312434.

    Article  PubMed  CAS  Google Scholar 

  9. 9.

    Oliveria S, Saraiya M, Geller A, Heneghan M, Jorgensen C. Sun exposure and risk of melanoma. Arch Dis Child. 2006;91(2):131–8. https://doi.org/10.1136/adc.2005.086918 PMC 2082713. PMID 16326797.

    Article  PubMed  CAS  Google Scholar 

  10. 10.

    Stolz W, et al. ABCD rule of dermatoscopy—A new practical method for early recognition of malignant-melanoma. Eur J Dermatol. 1994;4(7):521–7.

    Google Scholar 

  11. 11.

    Daniel Jensen J, Elewski BE. The ABCDEF Rule: Combining the "ABCDE Rule" and the "Ugly Duckling Sign" in an Effort to Improve Patient Self-Screening Examinations. J Clin Aesthet Dermatol. 2015;8(2):15 PMC 4345927. PMID 25741397.

    PubMed  PubMed Central  CAS  Google Scholar 

  12. 12.

    Friedman R, Rigel D, Kopf A. Early detection of malignant melanoma: the role of physician examination and self-examination of the skin. CA Cancer J Clin. 1985;35(3):130–51. https://doi.org/10.3322/canjclin.35.3.130 PMID 3921200.

    Article  PubMed  CAS  Google Scholar 

  13. 13.

    Bodenham D. A study of 650 observed malignant melanomas in the South-West region. Ann R Coll Surg Engl. 1968;43(4):218–39 PMC 2312310. PMID 5698493.

    PubMed  PubMed Central  CAS  Google Scholar 

  14. 14.

    Laennec RTH. Sur les melanoses. Bull Fac Med Paris. 1806;1:24–6.

    Google Scholar 

  15. 15.

    Norris W. Case of Fungoid Disease. Edinb Med Surg J. 1820;16(65):562–5 PMC 5830209. PMID 30332089.

    PubMed  PubMed Central  Google Scholar 

  16. 16.

    Cooper S (1844). The first lines of the theory and practice of surgery: including the principle operations. S.S. and W. Wood.

  17. 17.

    Cohen ME, Hudson DL, Banda PW, Blois MS. Neural network approach to detection of metastatic melanoma from chromatographic analysis of urine. Proc Annu Symp Comput Appl Med Care. 1991:295–9 PMID:1807608 PMCID: PMC2247542.

  18. 18.

    Ercal F, Chawla A, Stoecker WV, Lee HC, Moss RH. Neural network diagnosis of malignant melanoma from color images. IEEE Trans Biomed Eng. 1994;41(9):837–45 PMID: 7959811.

    Article  CAS  Google Scholar 

  19. 19.

    Brinker TJ, Hekler A, Enk AH, Klode J, Hauschild A, Berking C, Schilling B, Haferkamp S, Schadendorf D, Fröhling S, Utikal JS, von Kalle C, Collaborators. A convolutional neural network trained with dermoscopic images performed on par with 145 dermatologists in a clinical melanoma image classification task. Eur J Cancer. 2019;111:148–54. https://doi.org/10.1016/j.ejca.2019.02.005 Epub 2019 Mar 8. PMID:30852421.

    Article  PubMed  Google Scholar 

  20. 20.

    Haenssle HA, Fink C, Schneiderbauer R, Toberer F, Buhl T, Blum A, Kalloo A, Hassen ABH, Thomas L, Enk A, Uhlmann L, Reader study level-I and level-II Groups. Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists. Ann Oncol. 2018;29(8):1836–42. https://doi.org/10.1093/annonc/mdy166 PMID: 29846502.

    Article  PubMed  CAS  Google Scholar 

  21. 21.

    Xie F, Fan H, Li Y, Jiang Z, Meng R, Bovik A. Melanoma classification on Dermoscopy images using a neural network ensemble model. IEEE Trans Med Imaging. 2017;36(3):849–58. https://doi.org/10.1109/TMI.2016.2633551 Epub 2016 Dec 1. PMID:27913337.

    Article  PubMed  Google Scholar 

  22. 22.

    Nida N, Irtaza A, Javed A, Yousaf MH, Mahmood MT. Melanoma lesion detection and segmentation using deep region based convolutional neural network and fuzzy C-means clustering. Int J Med Inform. 2019;124:37–48. https://doi.org/10.1016/j.ijmedinf.2019.01.005 Epub 2019 Jan 18. PMID:30784425.

    Article  PubMed  Google Scholar 

  23. 23.

    Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, Thrun S. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542(7639):115–8. https://doi.org/10.1038/nature21056 Epub 2017 Jan 25. PMID: 28117445.

    Article  PubMed  CAS  Google Scholar 

  24. 24.

    Razmjooy N, Sheykhahmad FR, Ghadimi N. A Hybrid Neural Network - World Cup Optimization Algorithm for Melanoma Detection. Open Med (Wars). 2018;13:9–16. https://doi.org/10.1515/med-2018-0002 eCollection 2018. PMID: 29577090 PMCID: PMC5850997.

    Article  Google Scholar 

  25. 25.

    Roffman D, Hart G, Girardi M, Ko CJ, Deng J. Predicting non-melanoma skin cancer via a multi-parameterized artificial neural network. Sci Rep. 2018;8(1):1701. https://doi.org/10.1038/s41598-018-19907-9 PMID:29374196 PMCID: PMC5786038.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  26. 26.

    Gautam D, Ahmed M, Meena YK, Haq AU. Machine learning-based diagnosis of melanoma using macro images. Int J Numer Method Biomed Eng. 2018;34(5):e2953. https://doi.org/10.1002/cnm.2953 Epub 2018 Feb 20. PMID: 29266819.

    Article  PubMed  Google Scholar 

  27. 27.

    Brinker TJ, Hekler A, Enk AH, Klode J, Hauschild A, Berking C, Schilling B, Haferkamp S, Schadendorf D, Holland-Letz T, Utikal JS, von Kalle C, Collaborators. Deep learning outperformed 136 of 157 dermatologists in a head-to-head dermoscopic melanoma image classification task. Eur J Cancer. 2019;113:47–54. https://doi.org/10.1016/j.ejca.2019.04.001 Epub 2019 Apr 10. PMID: 30981091.

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

Not applicable.

About this supplement

This article has been published as part of BMC Bioinformatics Volume 21 Supplement 11, 2020: Selected Topics in “Systems Biology and Bioinformatics” - 2019: bioinformatics. The full contents of the supplement are available online at - https://bmcbioinformatics.biomedcentral.com/articles/supplements/volume-21-supplement-11.

Funding

The funding bodies played no role in the design of the study, research, writing and publication of the paper. Publication charges for this article have been funded by authors.

Author information

Affiliations

Authors

Contributions

Mila Efimenko, Alexander Ignatev, Konstantin Koshechkin. KK designed the study and analytical set-up. AI contributed to the project validation plan. ME performed publications search and selection. KK, and AI validated the methodology. ME conducted the analytical assessments. ME and AI interpreted the results. ME and KK wrote the manuscript. All authors have read and approved the present manuscript.

Corresponding author

Correspondence to Konstantin Koshechkin.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Efimenko, M., Ignatev, A. & Koshechkin, K. Review of medical image recognition technologies to detect melanomas using neural networks. BMC Bioinformatics 21, 270 (2020). https://doi.org/10.1186/s12859-020-03615-1

Download citation

Keywords

  • Melanoma classification
  • Skin cancer
  • Deep learning neural network
  • Convolutional neural network
  • Fuzzy clustering algorithm