Exploring simple siamese representation learning
To run unsupervised pre-training on ImageNet,
This is to do the unsupervised pre-training for 100 epochs. Please modify the path to your ImageNet data folder.
Note 1: I try to follow the setting in the paper, which is bs=512 and lr=0.1 on 8-GPU, but somehow I can not fit it. So I used the max batch_size that I can fit (432) while kept the lr unchaged (0.1).
Note 2: In pre-training, I didn’t fix the lr of prediction MLP. According to the paper (Table. 1), fixing the lr of prediction MLP can give slightly improvements (67.7% -> 68.1%). You can try it if interested.
To run linear evaluation,
The linear evaluation is done using NVIDIA LARC optimizer by setting
clip=False. The batch size is 4096.
Note: I first followed the setting in the paper, which is
Lr=0.32 (0.02*4096/256). But I can only got a result of 66.0%. Then I increased the learning rate to
Lr=1.6 (0.1*4096.256) and achieved the result of 67.8%. The results and models are given below.
|SimSiam||pretrained batchsize||lincls Lr||Top-1 Acc|
|Reproduced||432 (Model)||1.6||67.8% (Model)|
Thank Xinlei for his help on some implementation details.