Transformer models are good at capturing content-based global interactions, while CNNs exploit local features effectively. Conformer combine convolution neural networks and transformers to model both local and global dependencies of an audio sequence in a parameter-efficient way. Conformer significantly outperforms the previous Transformer and CNN based models achieving state-of-the-art accuracies.
This repository contains only model code, but you can train with conformer with this repository.
This project recommends Python 3.7 or higher.
We recommend creating a new virtual environment for this project (using virtual env or conda).
pip install numpy(Refer here for problem installing Numpy).
- Pytorch: Refer to PyTorch website to install the version w.r.t. your environment.
Install from source
Currently we only support installation from source code using setuptools. Checkout the source code and run the
pip install -e .
import torch import torch.nn as nn from conformer import Conformer batch_size, sequence_length, dim = 3, 12345, 80 cuda = torch.cuda.is_available() device = torch.device('cuda' if cuda else 'cpu') inputs = torch.rand(batch_size, sequence_length, dim).to(device) input_lengths = torch.IntTensor([12345, 12300, 12000]) targets = torch.LongTensor([[1, 3, 3, 3, 3, 3, 4, 5, 6, 2], [1, 3, 3, 3, 3, 3, 4, 5, 2, 0], [1, 3, 3, 3, 3, 3, 4, 2, 0, 0]]).to(device) target_lengths = torch.LongTensor([9, 8, 7]) model = nn.DataParallel(Conformer(num_classes=10, input_dim=dim, encoder_dim=32, num_encoder_layers=3, decoder_dim=32, device=device)).to(device) # Forward propagate outputs = model(inputs, input_lengths, targets, target_lengths) # Recognize input speech outputs = model.module.recognize(inputs, input_lengths)
Troubleshoots and Contributing
I appreciate any kind of feedback or contribution. Feel free to proceed with small issues like bug fixes, documentation improvement. For major contributions and new features, please discuss with the collaborators in corresponding issues.
I follow PEP-8 for code style. Especially the style of docstrings is important to generate documentation.