Pytorch port of Google Research's LEAF Audio paper published at ICLR 2021.
This port is not completely finished, but the Leaf() frontend is fully ported over, functional and validated to have similar outputs to the original tensorflow implementation. A few small things are missing, such as the SincNet and SincNet+ implementations, a few different pooling layers, etc.
PLEASE leave issues, pull requests, comments, or anything you find in using this repository that may be of value to others who will try to use this.
From the root directory of this repo, run:
pip install -e .
leaf_audio_pytorch mirrors it's original respository; imports and arguments are the same.
import leaf_audio_pytorch.frontend as frontend leaf = frontend.Leaf()
Installation for Developing
If you are looking to develop on this repo, the
requirements.txt contains everything needed to run the torch and tf
implementations of leaf audio simultaneously.
NOTE: There is some weird dependency stuff going on with the original
leaf-audio repo. Seems like its a
dependency issue with
waymo-open-dataset. These below commands are a workaround.
Install the packages required:
pip install -r requirements.txt --no-deps
leaf-audio repo from Git SSH:
pip install git+ssh://[email protected]/google-research/leaf-audio.git --no-deps
Then add the
leaf_audio_pytorch package as well
python setup.py develop
At this point everything should be good to go! The scripts in
test/ contains some testing code to validate the torch
implementation mirrors tf.
Some Things to Keep in Mind (PLEASE READ)
When writing this port, I ran a debugger of the torch and tf implementations side by side and validated that each layer
and operation mirrors the tensorflow implementation (to within a few significant digits, i.e. a tensor's values may variate
by 0.001). There is one notable exception: The depthwise convolution within the
GaussianLowpasspooling layer has
a larger variation in tensor values, but the ported operation still produces similar outputs. I'm not sure why this
operation is producing different values, but i'm currently
looking into it. Please do your own due diligence in using this port and making sure this works as expected.
March 29, I finished the initial version of the port, but I have not tested
Leaf()in a traning setting yet.
Leaf()throws no errors, meaning backprop works as expected. However, I do not
yet know how this will function during training.
As PyTorch and Tensorflow follow different tensor ordering conventions,
Leaf()does all of its operations and
outputs tensors with channels first.