Barlow
Barlow is a tool for identifying the failure modes for a given neural network. To achieve this, Barlow first creates a group of images such that all images in the given group have the same predicted class or the label. Then, it uses a (possibly separate) robust model to extract the embedding for all images in the group. The embedding encodes the human-interpretable visual attributes present in the image. It then learns a decision tree using this embedding that predicts failure on the neural network being inspected for failure.
Prerequisites
- Python 3.7+
- Pytorch 1.6+
- scikit-learn 0.23+
- scipy 1.5+
- matplotlib 3.2+
How to run?
- The robust model is taken from the robustness library and can be downloaded from this link in models/
- Run failure_explanation_sample.ipynb to identify example failure modes for the given set of images (in sample_data/) using the robust model located in models/.
Example failure mode
For the images with label water jug, when feature[1378] (visually identified as 'water jug handle') is less than 0.089335, error rate increases to 100.0% from 50.0%, i.e an increase of 50.0% in the failure rate.
Citation
@inproceedings{singlaCVPR2021,
title={Understanding Failures of Deep Networks via Robust Feature Extraction},
author={Sahil Singla and Besmira Nushi and Shital Shah and Ece Kamar and Eric Horvitz},
booktitle={The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
year={2021}
}
