Loss functions for image segmentation

A collection of loss functions for medical image segmentation

title = {Loss Odyssey in Medical Image Segmentation},
journal = {Medical Image Analysis},
volume = {71},
pages = {102035},
year = {2021},
author = {Jun Ma and Jianan Chen and Matthew Ng and Rui Huang and Yu Li and Chen Li and Xiaoping Yang and Anne L. Martel}
doi = {https://doi.org/10.1016/j.media.2021.102035},
url = {https://www.sciencedirect.com/science/article/pii/S1361841521000815}

Take-home message: compound loss functions are the most robust losses, especially for the highly imbalanced segmentation tasks.

Some recent side evidence: the winner in MICCAI 2020 HECKTOR Challenge used DiceFocal loss; the winner and runner-up in MICCAI 2020 ADAM Challenge used DiceTopK loss.

Date First Author Title Conference/Journal
20210418 Bingyuan Liu The hidden label-marginal biases of segmentation losses (pytorch) arxiv
20210330 Suprosanna Shit and Johannes C. Paetzold clDice - a Novel Topology-Preserving Loss Function for Tubular Structure Segmentation (keras and pytorch) CVPR 2021
20210325 Attila Szabo, Hadi Jamali-Rad Tilted Cross Entropy (TCE): Promoting Fairness in Semantic Segmentation arxiv
20210318 Xiaoling Hu Topology-Aware Segmentation Using Discrete Morse Theory arxiv ICLR 2021
20210211 Hoel Kervadec Beyond pixel-wise supervision: semantic segmentation with higher-order shape descriptors Submitted to MIDL 2021
20210210 Rosana EL Jurdi A Surprisingly Effective Perimeter-based Loss for Medical Image Segmentation Submitted to MIDL 2021
20201222 Zeju Li Analyzing Overfitting Under Class Imbalance in Neural Networks for Image Segmentation TMI
20210129 Nick Byrne A Persistent Homology-Based Topological Loss Function for Multi-class CNN Segmentation of Cardiac MRI arxiv STACOM 2020
20201019 Hyunseok Seo Closing the Gap Between Deep Neural Network Modeling and Biomedical Decision-Making Metrics in Segmentation via Adaptive Loss Functions TMI
20200929 Stefan Gerl A Distance-Based Loss for Smooth and Continuous Skin Layer Segmentation in Optoacoustic Images MICCAI 2020
20200821 Nick Byrne A persistent homology-based topological loss function for multi-class CNN segmentation of cardiac MRI arxiv STACOM
20200720 Boris Shirokikh Universal Loss Reweighting to Balance Lesion Size Inequality in 3D Medical Image Segmentation arxiv (pytorch) MICCAI 2020
20200708 Gonglei Shi Marginal loss and exclusion loss for partially supervised multi-organ segmentation (arXiv) MedIA
20200706 Yuan Lan An Elastic Interaction-Based Loss Function for Medical Image Segmentation (pytorch) (arXiv) MICCAI 2020
20200615 Tom Eelbode Optimization for Medical Image Segmentation: Theory and Practice when evaluating with Dice Score or Jaccard Index TMI
20200605 Guotai Wang Noise-robust Dice loss: A Noise-robust Framework for Automatic Segmentation of COVID-19 Pneumonia Lesions from CT Images (pytorch) TMI
202004 J. H. Moltz Contour Dice coefficient (CDC) Loss: Learning a Loss Function for Segmentation: A Feasibility Study ISBI
201912 Yuan Xue Shape-Aware Organ Segmentation by Predicting Signed Distance Maps (arxiv) (pytorch) AAAI 2020
201912 Xiaoling Hu Topology-Preserving Deep Image Segmentation (paper) (pytorch) NeurIPS
201910 Shuai Zhao Region Mutual Information Loss for Semantic Segmentation (paper) (pytorch) NeurIPS 2019
201910 Shuai Zhao Correlation Maximized Structural Similarity Loss for Semantic Segmentation (paper) arxiv
201908 Pierre-AntoineGanaye Removing Segmentation Inconsistencies with Semi-Supervised Non-Adjacency Constraint (paper) (official pytorch) Medical Image Analysis
201906 Xu Chen Learning Active Contour Models for Medical Image Segmentation (paper) (official-keras) CVPR 2019
20190422 Davood Karimi Reducing the Hausdorff Distance in Medical Image Segmentation with Convolutional Neural Networks (paper) (pytorch) TMI 201907
20190417 Francesco Caliva Distance Map Loss Penalty Term for Semantic Segmentation (paper) MIDL 2019
20190411 Su Yang Major Vessel Segmentation on X-ray Coronary Angiography using Deep Networks with a Novel Penalty Loss Function (paper) MIDL 2019
20190405 Boah Kim Multiphase Level-Set Loss for Semi-Supervised and Unsupervised Segmentation with Deep Learning (paper) arxiv
201901 Seyed Raein Hashemi Asymmetric Loss Functions and Deep Densely Connected Networks for Highly Imbalanced Medical Image Segmentation: Application to Multiple Sclerosis Lesion Detection (paper) IEEE Access
201812 Hoel Kervadec Boundary loss for highly unbalanced segmentation (paper), (pytorch 1.0) MIDL 2019
201810 Nabila Abraham A Novel Focal Tversky loss function with improved Attention U-Net for lesion segmentation (paper) (keras) ISBI 2019
201809 Fabian Isensee CE+Dice: nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation (paper) arxiv
20180831 Ken C. L. Wong 3D Segmentation with Exponential Logarithmic Loss for Highly Unbalanced Object Sizes (paper) MICCAI 2018
20180815 Wentao Zhu Dice+Focal: AnatomyNet: Deep Learning for Fast and Fully Automated Whole-volume Segmentation of Head and Neck Anatomy (arxiv) (pytorch) Medical Physics
201806 Javier Ribera Weighted Hausdorff Distance: Locating Objects Without Bounding Boxes (paper), (pytorch) CVPR 2019
201805 Saeid Asgari Taghanaki Combo Loss: Handling Input and Output Imbalance in Multi-Organ Segmentation (arxiv) (keras) Computerized Medical Imaging and Graphics
201709 S M Masudur Rahman AL ARIF Shape-aware deep convolutional neural network for vertebrae segmentation (paper) MICCAI 2017 Workshop
201708 Tsung-Yi Lin Focal Loss for Dense Object Detection (paper), (code) ICCV, TPAMI
20170711 Carole Sudre Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations (paper) DLMIA 2017
20170703 Lucas Fidon Generalised Wasserstein Dice Score for Imbalanced Multi-class Segmentation using Holistic Convolutional Networks (paper) MICCAI 2017 BrainLes
201705 Maxim Berman The Lovász-Softmax loss: A tractable surrogate for the optimization of the intersection-over-union measure in neural networks (paper), (code) CVPR 2018
201701 Seyed Sadegh Mohseni Salehi Tversky loss function for image segmentation using 3D fully convolutional deep networks (paper) MICCAI 2017 MLMI
201612 Md Atiqur Rahman Optimizing Intersection-Over-Union in Deep Neural Networks for Image Segmentation (paper) 2016 International Symposium on Visual Computing
201608 Michal Drozdzal "Dice Loss (without square)" The Importance of Skip Connections in Biomedical Image Segmentation (arxiv) DLMIA 2016
201606 Fausto Milletari "Dice Loss (with square)" V-net: Fully convolutional neural networks for volumetric medical image segmentation (arxiv), (caffe code) International Conference on 3D Vision
201605 Zifeng Wu TopK loss Bridging Category-level and Instance-level Semantic Image Segmentation (paper) arxiv
201511 Tom Brosch "Sensitivity-Specifity loss" Deep Convolutional Encoder Networks for Multiple Sclerosis Lesion Segmentation (paper) (code) MICCAI 2015
201505 Olaf Ronneberger "Weighted cross entropy" U-Net: Convolutional Networks for Biomedical Image Segmentation (paper) MICCAI 2015
201309 Gabriela Csurka What is a good evaluation measure for semantic segmentation? (paper) BMVA 2013

Most of the corresponding tensorflow code can be found here.