Empirical Experiments in "Feature Learning in Infinite-width Neural Networks"
This repo contains code to replicate our experiments (Word2Vec, MAML) in our paper
Feature Learning in Infinite-Width Neural Networks
Greg Yang, Edward Hu
In short, the code here will allow you to train feature learning infinite-width neural networks on Word2Vec and on Omniglot (via MAML).
Our results on Word2Vec:
Our Results on MAML:
Please see the README in individual folders for more details.
This is the 4th paper in the Tensor Programs series ([0][1][2][3]). Also see here for code in previous papers for calculating the GP and NTK limits of wide neural networks.
