[TensorFlow 2] A Simple Baseline for Bayesian Uncertainty in Deep Learning: SWA-Gaussian (SWAG)
TensorFlow implementation of “A Simple Baseline for Bayesian Uncertainty in Deep Learning”
Concept
Algorithm to utilize the SWAG [1].
Equation for the weight sampling from SWAG [1].
Results
The red color and the blue color represent the initial state and current state respectively.
| Variable | MNIST | CIFAR10 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Performance
MNIST
| Method | Accuracy | Precision | Recall | F1-Score |
|---|---|---|---|---|
| Final Epoch | 0.99230 | 0.99231 | 0.99222 | 0.99226 |
| Best Loss | 0.99350 | 0.99350 | 0.99338 | 0.99344 |
| SWAG (S = 30) | 0.99310 | 0.99305 | 0.99299 | 0.99302 |
| SWAG (Last Momentum) | 0.99340 | 0.99340 | 0.99330 | 0.99335 |
CIFAR10
| Method | Accuracy | Precision | Recall | F1-Score |
|---|---|---|---|---|
| Final Epoch | 0.73130 | 0.73349 | 0.73130 | 0.73147 |
| Best Loss | 0.73240 | 0.73205 | 0.73240 | 0.73099 |
| SWAG (S = 30) | 0.74100 | 0.74622 | 0.74100 | 0.74260 |
| SWAG (Last Momentum) | 0.73490 | 0.73888 | 0.73490 | 0.73561 |
Requirements
- Python 3.7.6
- Tensorflow 2.3.0
- Numpy 1.18.15
- whiteboxlayer 0.1.15
Reference
[1] Wesley Maddox et al. (2019). A Simple Baseline for Bayesian Uncertainty in Deep Learning. arXiv preprint arXiv:1902.02476.
