CDS Guest Editorial: Improving Weakly Supervised Lesion Segmentation using Multi-Task Learning

This entry is a part of the NYU Center for Data Science blog’s recurring guest editorial series. Tianshu Chu and Xinmeng Li are CDS MS students. Huy V. Vo is a PhD student from INRIA and Valeo.ai. Dr. Ronald M. Summers is a senior investigator in the Imaging Biomarkers and Computer-Aided Diagnosis Laboratory at the NIH Clinical Center. Dr. Elena Sizikova is a Moore-Sloan CDS Faculty fellow.

From the “Improving Weakly Supervised Lesion Segmentation Using Multi-Task Learning” paper: visualization of lesion groups obtained by clustering ellipse measurements. In a dataset, each row corresponds to a cluster. Clustering groups similar examples within each of the considered CT (DeepLesion, LiTS) and dermatoscopic (HAM10K) datasets.

Two CDS masters students, Tianshu Chu and Xinmeng Li, Huy V. Vo, a PhD student from INRIA and Valeo.ai, and Dr. Elena Sizikova, a CDS Faculty fellow, recently collaborated with Dr. Ronald. M. Summers from the NIH Clinical Center on “Improving Weakly Supervised Lesion Segmentation Using Multi-Task Learning”. Assessing lesion shape and growth is an important problem in medical imaging research, and automating this process helps reduce processing time and highlights important oncology properties for radiologists.

The project will be presented at the upcoming Medical Imaging with Deep Learning (MIDL) 2021 Conference on July 8th and 9th. MIDL is a conference focused on the development of deep learning approaches to analysis of medical images.

The proposed method explains how to obtain accurate segmentation of lesions in computed tomography (CT) scan images and also dermatoscopic (skin) lesion images with cheap and readily available annotations. Deep learning-based segmentation methods achieve very high segmentation accuracies on medical imaging tasks (Ronneberger et al., 2015), however, they require large pixel-level annotated training datasets which are often not available or too costly to obtain. The paper shows that combining weak annotations, i.e., response evaluation criteria in solid tumors (RECIST), which are ellipse-shaped annotations used by radiologists in archival systems, with multi-task learning (MTL) allows one to quickly train a neural network without pixel-level training data. Compared to previous approaches that tackled this task, the proposed method is more accurate and faster to train.

More generally, Tianshu Chu, Xinmeng Li, and Huy V. Vo are interested in developing accurate, automatic image-processing techniques. Dr. Elena Sizikova develops computer vision tools for analysis of biomedical images.

For additional information, please visit the paper’s project page.

This project was supported by the Moore-Sloan Data Science Environment initiative (funded by Alfred P. Sloan Foundation and the Gordon and Betty Moore Foundation) through the NYU Center for Data Science (Tianshu Chu, Xinmeng Li, Dr. Elena Sizikova). Huy V. Vo was supported in part by the Inria/NYU collaboration, the French government under management of Agence Nationale de la Recherche as part of the “Investissements d’avenir” program, reference ANR19-P3IA-0001 (PRAIRIE 3IA Institute), and a Valeo/PRAIRIE CIFRE Fellowship. Dr. Ronald M. Summers was supported by the Intramural Research Program of the National Institutes of Health Clinical Center.

References

1. Ronneberger, Olaf, Philipp Fischer, and Thomas Brox. “U-net: Convolutional networks for biomedical image segmentation.” International Conference on Medical image computing and computer-assisted intervention. Springer, Cham, 2015.

By Tianshu Chu, Xinmeng Li, Huy V. Vo, Ronald M. Summers, and Elena Sizikova