Easy to use Audio Tagging in PyTorch

Download the dataset and prepare it into the following structure.

datasets
|__ ESC50
    |__ audio

|__ Urbansound8k
    |__ audio

|__ FSDKaggle2018
    |__ audio_train
    |__ audio_test
    |__ FSDKaggle2018.meta
        |__ train_post_competition.csv
        |__ test_post_competition_scoring_clips.csv

|__ SpeechCommandsv1/v2
    |__ bed
    |__ bird
    |__ ...
    |__ testing_list.txt
    |__ validation_list.txt

Currently, the following augmentations are supported. More will be added in the future. You can test the effects of augmentations with this notebook

WaveForm Augmentations:

• [x] MixUp
• [x] Background Noise
• [x] Gaussian Noise
• [x] Fade In/Out
• [x] Volume
• [ ] CutMix

Spectrogram Augmentations:

• [x] Time Masking
• [x] Frequency Masking
• [x] Filter Augmentation

• python >= 3.6
• pytorch >= 1.8.1
• torchaudio >= 0.8.1

Other requirements can be installed with pip install -r requirements.txt.

• Create a configuration file in configs. Sample configuration for ESC50 dataset can be found here.
• Copy the contents of this and then edit the fields you think if it is needed.
• This configuration file is needed for all of training, evaluation and prediction scripts.

To train with a single GPU:

$ python tools/train.py --cfg configs/CONFIG_FILE_NAME.yaml

To train with multiple gpus, set DDP field in config file to true and run as follows:

$ python -m torch.distributed.launch --nproc_per_node=2 --use_env tools/train.py --cfg configs/CONFIG_FILE_NAME.yaml

Make sure to set MODEL_PATH of the configuration file to your trained model directory.

$ python tools/val.py --cfg configs/CONFIG_FILE.yaml

• Set MODEL_PATH of the configuration file to your model's trained weights.
• Change the dataset name in DATASET >> NAME as your trained model's dataset.
• Set the testing audio file path in TEST >> FILE.
• Run the following command.

$ python tools/infer.py --cfg configs/CONFIG_FILE.yaml

## for example
$ python tools/infer.py --cfg configs/audioset.yaml

You will get an output similar to this:

Class                     Confidence
----------------------  ------------
Speech                     0.897762
Telephone bell ringing     0.752206
Telephone                  0.219329
Inside, small room         0.20761
Music                      0.0770325

• Set MODEL_PATH of the configuration file to your model's trained weights.
• Change the dataset name in DATASET >> NAME as your trained model's dataset.
• Set the testing audio file path in TEST >> FILE.
• Run the following command.

$ python tools/sed_infer.py --cfg configs/CONFIG_FILE.yaml

## for example
$ python tools/sed_infer.py --cfg configs/audioset_sed.yaml

You will get an output similar to this:

Class                     Start    End
----------------------  -------  -----
Speech                      2.2    7
Telephone bell ringing      0      2.5

The following plot will also be shown, if you set PLOT to true:

• https://github.com/qiuqiangkong/audioset_tagging_cnn
• https://github.com/YuanGongND/ast
• https://github.com/frednam93/FilterAugSED
• https://github.com/lRomul/argus-freesound

@misc{kong2020panns,
      title={PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition}, 
      author={Qiuqiang Kong and Yin Cao and Turab Iqbal and Yuxuan Wang and Wenwu Wang and Mark D. Plumbley},
      year={2020},
      eprint={1912.10211},
      archivePrefix={arXiv},
      primaryClass={cs.SD}
}

@misc{gong2021ast,
      title={AST: Audio Spectrogram Transformer}, 
      author={Yuan Gong and Yu-An Chung and James Glass},
      year={2021},
      eprint={2104.01778},
      archivePrefix={arXiv},
      primaryClass={cs.SD}
}

@misc{nam2021heavily,
      title={Heavily Augmented Sound Event Detection utilizing Weak Predictions}, 
      author={Hyeonuk Nam and Byeong-Yun Ko and Gyeong-Tae Lee and Seong-Hu Kim and Won-Ho Jung and Sang-Min Choi and Yong-Hwa Park},
      year={2021},
      eprint={2107.03649},
      archivePrefix={arXiv},
      primaryClass={eess.AS}
}