A study has found that a direct-to-consumer machine model for detecting skin cancers incorrectly classified rare and aggressive cancers as low-risk.
The study, presented at the 30th European Academy of Dermatology Congress (EADV), suggests that making these apps shouldn’t be available directly to the public without transparency on performance metrics regarding rare but potentially life-threatening skin cancers is ethically questionable.
Researchers focused on two skin cancers, Merkel cell carcinoma (MCC) and amelanotic melanoma. They created a dataset of 116 images of these rare cancers and of the benign lesions and assessed these images with two machine-learning models.
The first model studied was a certified medical device, sold to the public via the App store and advertised as being able to diagnose 95% of skin cancers. The second model was available for research purposes only.
The results showed that the first model incorrectly classified 17.9% of MCCs and 22.9% of amelanotic melanomas as low-risk. 62.2% of benign lesions were classified as high-risk. For the second model, MCC was not included in the top five diagnosis for any of the 28 MCC images analysed, raising the possibility that the model had not been trained that this disease class exists.
Marie-Aleth Richard, EADV board member,
commented, “The number of skin cancer detection apps available for consumer use
is growing, but as demonstrated in this research, there must be more
transparency around the safety and efficacy of these apps. Failure to be
transparent could put lives at risk.”
