FixRes

FixRes is a simple method for fixing the train-test resolution discrepancy. It can improve the performance of any convolutional neural network architecture.

This repository reproduces the results of the paper: "Fixing the train-test resolution discrepancy"

The method is described in "Fixing the train-test resolution discrepancy" (arXiv link).

BibTeX reference to cite, if you use it:

@ARTICLE{2019arXivFixRes,
       author = {Hugo, Touvron and Vedaldi, Andrea and Douze, Matthijs and J{\'e}gou, Herv{\'e}},
        title = "{Fixing the train-test resolution discrepancy}",
      journal = {arXiv e-prints},
         year = "2019",
        month = "June",
}

Installation

The FixRes code requires

• Python 3.6 or higher
• PyTorch 1.0 or higher

and the requirements highlighted inrequirements.txt (for Anaconda)

Cluster settings

Ours codes were executed on a cluster with several GPUs. As configurations are different from one cluster to another, we provide a generic implementation. You must run the code on each GPU by specifying job-id, local-rank, global-rank, and num-tasks which is not very convenient. Therefore, we strongly recommend to adapt our code according to the configuration of your cluster.

Using the code

The configurations given in the examples provide the results of the Pretrained Networks table (Table 2 in the article).
The trainning and fine-tuning codes record the learned model in a checkpoint.pth file.

Extracting features with pre-trained networks

Pre-trained networks

We provide pre-trained networks with differents trunks, we report in the table validation resolution, Top-1 and Top-5 accuracy on ImageNet validation set:

(*) We use Horizontal flip, shifted Center Crop and color jittering for fine-tuning (described in transforms_v2.py)

To load a network, use the following PyTorch code:

import torch
from .resnext_wsl import resnext101_32x48d_wsl

model=resnext101_32x48d_wsl(progress=True) # example with the ResNeXt-101 32x48d 

pretrained_dict=torch.load('ResNeXt101_32x48d.pth',map_location='cpu')['model']

model_dict = model.state_dict()
for k in model_dict.keys():
    if(('module.'+k) in pretrained_dict.keys()):
        model_dict[k]=pretrained_dict.get(('module.'+k))
model.load_state_dict(model_dict)

The network takes images in any resolution.
A normalization pre-processing step is used, with mean [0.485, 0.456, 0.406].
and standard deviation [0.229, 0.224, 0.225] for ResNet-50 and ResNeXt-101 32x48d,
use mean [0.5, 0.5, 0.5] and standard deviation [0.5, 0.5, 0.5] with PNASNet.
You can find the code in transforms.py.

Features extracted from the ImageNet validation set

We provide the probabilities, embedding and labels of each image in the ImageNet validation so that the results can be reproduced easily.

Embedding files are matrixes of size 50000 by 2048 for all models except for PNASNet where the size is 50000 by 4320, embeddings are extracted after the last spatial pooling. The softmax are matrixes of sizes 50000 by 1000 it representing the probability of each class for each image.

(*) We use Horizontal flip, shifted Center Crop and color jittering for fine-tuning (described in transforms_v2.py)

Evaluation of the networks

See help (-h flag) for detailed parameter list of each script before executing the code.

Classification results

main_evaluate_imnet.py evaluates the network on standard benchmarks.

main_evaluate_softmax.py evaluates the network on ImageNet-val with already extracted softmax output. (Much faster to execute)

Example evaluation procedure

# FixResNeXt-101 32x48d
python main_evaluate_imnet.py --input-size 320 --architecture 'IGAM_Resnext101_32x48d' --weight-path 'ResNext101_32x48d.pth'
# FixResNet-50
python main_evaluate_imnet.py --input-size 384 --architecture 'ResNet50' --weight-path 'ResNet50.pth'

#FixPNASNet-5
python main_evaluate_imnet.py --input-size 480 --architecture 'PNASNet' --weight-path 'PNASNet.pth'

The following code give results that corresponds to table 2 in the paper :

# FixResNeXt-101 32x48d
python main_evaluate_softmax.py --architecture 'IGAM_Resnext101_32x48d' --save-path 'where_softmax_and_labels_are_saved'

# FixPNASNet-5
python main_evaluate_softmax.py --architecture 'PNASNet' --save-path 'where_softmax_and_labels_are_saved'

# FixResNet50
python main_evaluate_softmax.py --architecture 'ResNet50' --save-path 'where_softmax_and_labels_are_saved'

Features extraction

main_extract.py exctrat embedding, labels and probability with the networks.

Example extraction procedure

# FixResNeXt-101 32x48d
python main_extract.py --input-size 320 --architecture 'IGAM_Resnext101_32x48d' --weight-path 'ResNeXt101_32x48d.pth' --save-path 'where_output_will_be_save'
# FixResNet-50
python main_extract.py --input-size 384 --architecture 'ResNet50' --weight-path 'ResNet50.pth' --save-path 'where_output_will_be_save'

# FixPNASNet-5
python main_extract.py --input-size 480 --architecture 'PNASNet' --weight-path 'PNASNet.pth' --save-path 'where_output_will_be_save'

Fine-tuning existing network with our Method

See help (-h flag) for detailed parameter list of each script before executing the code.

Classifier and Batch-norm fine-tuning

main_finetune.py fine-tune the network on standard benchmarks.

Example fine-tuning procedure

# FixResNeXt-101 32x48d
python main_finetune.py --input-size 320 --architecture 'IGAM_Resnext101_32x48d' --epochs 1 --batch 8 --num-tasks 32 --learning-rate 1e-3

# FixResNet-50
python main_finetune.py --input-size 384 --architecture 'ResNet50' --epochs 56 --batch 64 --num-tasks 8 --learning-rate 1e-3

# FixPNASNet-5
python main_finetune.py --input-size 480 --architecture 'PNASNet' --epochs 1 --batch 64 --num-tasks 8 --learning-rate 1e-4

Using transforms_v2 for fine-tuning

To reproduce our best results we must use the data-augmentation of transforms_v2 and use almost the same parameters as for the classic data augmentation, the only changes are the learning rate which must be 1e-4 and the number of epochs which must be 11. For FixResNet-50 fine-tune you have to use 31 epochs and a learning rate of 1e-3 and for FixResNet-50 CutMix you have to use 11 epochs and a learning rate of 1e-3.
Here is how to use transforms_v2 :

from torchvision import datasets
from .transforms_v2 import get_transforms

transform = get_transforms(input_size=Train_size,test_size=Test_size, kind='full', crop=True, need=('train', 'val'), backbone=None)
train_set = datasets.ImageFolder(train_path,transform=transform['val_train'])
test_set = datasets.ImageFolder(val_path,transform=transform['val_test'])

Training

See help (-h flag) for detailed parameter list of each script before executing the code.

Train ResNet-50 from scratch

main_resnet50_scratch.py Train ResNet-50 on standard benchmarks.

Example training procedure

# ResNet50
python main_resnet50_scratch.py --batch 64 --num-tasks 8 --learning-rate 2e-2

GitHub