A Text-to-Speech Transformer in TensorFlow 2.
Implementation of an autoregressive Transformer based neural network for Text-to-Speech (TTS).
This repo is based on the following paper:
Spectrograms produced with LJSpeech and standard data configuration from this repo are compatible with WaveRNN.
These samples' spectrograms are converted using the pre-trained WaveRNN vocoder.
The TTS weights used for these samples can be found here.
Check out the notebooks folder for predictions with TransformerTTS and WaveRNN or just try out our Colab notebook:
Make sure you have:
- Python >= 3.6
Install espeak as phonemizer backend (for macOS use brew):
sudo apt-get install espeak
Then install the rest with pip:
pip install -r requirements.txt
Read the individual scripts for more command line arguments.
You can directly use LJSpeech to create the training dataset.
- If training LJSpeech, or if unsure, simply use
- EDIT PATHS: in
data_config.yamledit the paths to point at your dataset and log folders
Prepare a dataset in the following format:
|- dataset_folder/ | |- metadata.csv | |- wav/ | |- file1.wav | |- ...
metadata.csv has the following format:
Create training dataset
python create_dataset.py --config config/standard
python train.py --config config/standard
Training & Model configuration
- Training and model settings can be configured in
Resume or restart training
- To resume training simply use the same configuration files AND
--session_nameflag, if any
- To restart training, delete the weights and/or the logs from the logs folder with the training flag
We log some information that can be visualized with TensorBoard:
tensorboard --logdir /logs/directory/
from utils.config_manager import ConfigManager from utils.audio import reconstruct_waveform config_loader = ConfigManager('config/standard') model = config_loader.load_model() out = model.predict('Please, say something.') # Convert spectrogram to wav (with griffin lim) wav = reconstruct_waveform(out['mel'].numpy().T, config=config_loader.config)
- Francesco Cardinale, github: cfrancesco