Using deep learning for dermatologist-level detection of suspicious pigmented skin lesions from wide-field images

Luis R. Soenksen; Timothy Kassis; Susan T. Conover; Berta Marti-Fuster; Judith S. Birkenfeld; Jason Tucker-Schwartz; Asif Naseem; Robert R. Stavert; Caroline C. Kim; Maryanne M. Senna; José Avilés-Izquierdo; James J. Collins; Regina Barzilay; Martha L. Gray

doi:10.1126/scitranslmed.abb3652

Research ArticleCancer

Using deep learning for dermatologist-level detection of suspicious pigmented skin lesions from wide-field images

View ORCID ProfileLuis R. Soenksen ¹,²,³,⁴,⁵,*,
Timothy Kassis⁶,
View ORCID ProfileSusan T. Conover²,
Berta Marti-Fuster ²,⁵,
View ORCID ProfileJudith S. Birkenfeld ²,⁵,
View ORCID ProfileJason Tucker-Schwartz²,⁵,
Asif Naseem²,⁵,
View ORCID ProfileRobert R. Stavert⁷,⁸,⁹,
Caroline C. Kim¹⁰,¹¹,
View ORCID ProfileMaryanne M. Senna⁹,¹²,
View ORCID ProfileJosé Avilés-Izquierdo¹³,
View ORCID ProfileJames J. Collins²,³,⁴,⁶,¹⁴,¹⁵,
View ORCID ProfileRegina Barzilay¹⁶,¹⁷ and
View ORCID ProfileMartha L. Gray ²,⁴,⁵,¹⁷

¹Department of Mechanical Engineering, Massachusetts Institute of Technology, 77 Massachusetts Ave, Cambridge, MA 02139, USA.
²Institute for Medical Engineering and Science, Massachusetts Institute of Technology, 77 Massachusetts Ave, Cambridge, MA 02139, USA.
³Wyss Institute for Biologically Inspired Engineering, Harvard University, 3 Blackfan Cir, Boston, MA 02115, USA.
⁴Harvard-MIT Program in Health Sciences and Technology, Cambridge, MA 02139, USA.
⁵MIT linQ, Massachusetts Institute of Technology Cambridge, MA 02148, USA.
⁶Department of Biological Engineering, Massachusetts Institute of Technology, 77 Massachusetts Ave, Cambridge, 02139, MA, USA.
⁷Division of Dermatology, Cambridge Health Alliance, 1493 Cambridge Street, Cambridge, MA 02139, USA.
⁸Department of Dermatology, Beth Israel Deaconess Medical Center, 330 Brookline Ave, Boston, MA 02215, USA.
⁹Department of Dermatology, Harvard Medical School, 25 Shattuck St, Boston, MA 02115, USA.
¹⁰Pigmented Lesion Program, Newton Wellesley Dermatology Associates, 65 Walnut Street Suite 520 Wellesley Hills, MA 02481, USA.
¹¹Department of Dermatology, Tufts Medical Center, 260 Tremont Street Biewend Building, Boston, MA 02116, USA.
¹²Department of Dermatology, Massachusetts General Hospital, 55 Fruit St, Boston, MA 02114
¹³Department of Dermatology, Hospital General Universitario Gregorio Marañón, Calle del Dr. Esquerdo 46, 28007 Madrid, Spain.
¹⁴Broad Institute of MIT and Harvard, Cambridge, MA 02142, USA.
¹⁵School of Engineering and Applied Sciences, Harvard University, Cambridge, MA 02138, USA.
¹⁶Computer Science and Artificial Intelligence Lab, Massachusetts Institute of Technology, 77 Massachusetts Ave, Cambridge, MA 02139, USA.
¹⁷Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology Cambridge, MA 02148, USA.

↵*Corresponding author. Email: soenksen{at}mit.edu

See allHide authors and affiliations

Science Translational Medicine 17 Feb 2021:
Vol. 13, Issue 581, eabb3652
DOI: 10.1126/scitranslmed.abb3652

Luis R. Soenksen

1Department of Mechanical Engineering, Massachusetts Institute of Technology, 77 Massachusetts Ave, Cambridge, MA 02139, USA.

2Institute for Medical Engineering and Science, Massachusetts Institute of Technology, 77 Massachusetts Ave, Cambridge, MA 02139, USA.

3Wyss Institute for Biologically Inspired Engineering, Harvard University, 3 Blackfan Cir, Boston, MA 02115, USA.

4Harvard-MIT Program in Health Sciences and Technology, Cambridge, MA 02139, USA.

5MIT linQ, Massachusetts Institute of Technology Cambridge, MA 02148, USA.

Find this author on Google Scholar
Find this author on PubMed
Search for this author on this site
ORCID record for Luis R. Soenksen
For correspondence: soenksen@mit.edu

Timothy Kassis

6Department of Biological Engineering, Massachusetts Institute of Technology, 77 Massachusetts Ave, Cambridge, 02139, MA, USA.

Find this author on Google Scholar
Find this author on PubMed
Search for this author on this site

Susan T. Conover

2Institute for Medical Engineering and Science, Massachusetts Institute of Technology, 77 Massachusetts Ave, Cambridge, MA 02139, USA.

Find this author on Google Scholar
Find this author on PubMed
Search for this author on this site
ORCID record for Susan T. Conover

Berta Marti-Fuster

2Institute for Medical Engineering and Science, Massachusetts Institute of Technology, 77 Massachusetts Ave, Cambridge, MA 02139, USA.

5MIT linQ, Massachusetts Institute of Technology Cambridge, MA 02148, USA.

Find this author on Google Scholar
Find this author on PubMed
Search for this author on this site

Judith S. Birkenfeld

2Institute for Medical Engineering and Science, Massachusetts Institute of Technology, 77 Massachusetts Ave, Cambridge, MA 02139, USA.

5MIT linQ, Massachusetts Institute of Technology Cambridge, MA 02148, USA.

Find this author on Google Scholar
Find this author on PubMed
Search for this author on this site
ORCID record for Judith S. Birkenfeld

Jason Tucker-Schwartz

2Institute for Medical Engineering and Science, Massachusetts Institute of Technology, 77 Massachusetts Ave, Cambridge, MA 02139, USA.

5MIT linQ, Massachusetts Institute of Technology Cambridge, MA 02148, USA.

Find this author on Google Scholar
Find this author on PubMed
Search for this author on this site
ORCID record for Jason Tucker-Schwartz

Asif Naseem

2Institute for Medical Engineering and Science, Massachusetts Institute of Technology, 77 Massachusetts Ave, Cambridge, MA 02139, USA.

5MIT linQ, Massachusetts Institute of Technology Cambridge, MA 02148, USA.

Find this author on Google Scholar
Find this author on PubMed
Search for this author on this site

Robert R. Stavert

7Division of Dermatology, Cambridge Health Alliance, 1493 Cambridge Street, Cambridge, MA 02139, USA.

8Department of Dermatology, Beth Israel Deaconess Medical Center, 330 Brookline Ave, Boston, MA 02215, USA.

9Department of Dermatology, Harvard Medical School, 25 Shattuck St, Boston, MA 02115, USA.

Find this author on Google Scholar
Find this author on PubMed
Search for this author on this site
ORCID record for Robert R. Stavert

Caroline C. Kim

10Pigmented Lesion Program, Newton Wellesley Dermatology Associates, 65 Walnut Street Suite 520 Wellesley Hills, MA 02481, USA.

11Department of Dermatology, Tufts Medical Center, 260 Tremont Street Biewend Building, Boston, MA 02116, USA.

Find this author on Google Scholar
Find this author on PubMed
Search for this author on this site

Maryanne M. Senna

9Department of Dermatology, Harvard Medical School, 25 Shattuck St, Boston, MA 02115, USA.

12Department of Dermatology, Massachusetts General Hospital, 55 Fruit St, Boston, MA 02114

Find this author on Google Scholar
Find this author on PubMed
Search for this author on this site
ORCID record for Maryanne M. Senna

José Avilés-Izquierdo

13Department of Dermatology, Hospital General Universitario Gregorio Marañón, Calle del Dr. Esquerdo 46, 28007 Madrid, Spain.

Find this author on Google Scholar
Find this author on PubMed
Search for this author on this site
ORCID record for José Avilés-Izquierdo

James J. Collins

2Institute for Medical Engineering and Science, Massachusetts Institute of Technology, 77 Massachusetts Ave, Cambridge, MA 02139, USA.

3Wyss Institute for Biologically Inspired Engineering, Harvard University, 3 Blackfan Cir, Boston, MA 02115, USA.

4Harvard-MIT Program in Health Sciences and Technology, Cambridge, MA 02139, USA.

6Department of Biological Engineering, Massachusetts Institute of Technology, 77 Massachusetts Ave, Cambridge, 02139, MA, USA.

14Broad Institute of MIT and Harvard, Cambridge, MA 02142, USA.

15School of Engineering and Applied Sciences, Harvard University, Cambridge, MA 02138, USA.

Find this author on Google Scholar
Find this author on PubMed
Search for this author on this site
ORCID record for James J. Collins

Regina Barzilay

16Computer Science and Artificial Intelligence Lab, Massachusetts Institute of Technology, 77 Massachusetts Ave, Cambridge, MA 02139, USA.

17Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology Cambridge, MA 02148, USA.

Find this author on Google Scholar
Find this author on PubMed
Search for this author on this site
ORCID record for Regina Barzilay

Martha L. Gray

2Institute for Medical Engineering and Science, Massachusetts Institute of Technology, 77 Massachusetts Ave, Cambridge, MA 02139, USA.

4Harvard-MIT Program in Health Sciences and Technology, Cambridge, MA 02139, USA.

5MIT linQ, Massachusetts Institute of Technology Cambridge, MA 02148, USA.

17Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology Cambridge, MA 02148, USA.

Find this author on Google Scholar
Find this author on PubMed
Search for this author on this site
ORCID record for Martha L. Gray

Article
Figures & Data
Info & Metrics
eLetters
PDF

You are currently viewing the editor's summary.

View Full Text

Log in to view the full text

via AAAS login

AAAS login provides access to Science for AAAS members, and access to other journals in the Science family to users who have purchased individual subscriptions.

Become an AAAS Member
Activate your Account
Purchase Access to Other Journals in the Science Family
Account Help

Log in through your institution

More options

Purchase digital access to this article

Download and print this article for your personal scholarly, research, and educational use.

Purchase this issue in print

Buy a single issue of Science for just $15 USD.

Finding the odd one out

Early identification of skin cancer is key to improving patient outcome. Soenksen et al. built a deep convolutional neural network that examines lesions from a given patient present in wide-field images, including those taken with cell phone cameras. Rather than evaluate a single lesion at a time looking for predetermined signs of neoplasia, the algorithm identifies lesions that differ from most of the other marks on that patient’s skin, flagging them for further examination and ranking them in order of suspiciousness. The algorithm performed similarly to board-certified dermatologists and could potentially be used at primary care visits to help clinicians triage suspicious lesions for follow-up.

https://www.sciencemag.org/about/science-licenses-journal-article-reuse

This is an article distributed under the terms of the Science Journals Default License.

View Full Text

Using deep learning for dermatologist-level detection of suspicious pigmented skin lesions from wide-field images

Finding the odd one out

Science Translational Medicine

Article Tools