PortaSpeech – PyTorch Implementation

PyTorch Implementation of PortaSpeech: Portable and High-Quality Generative Text-to-Speech.

Model Size

Module Normal Small Normal (paper) Small (paper)
Total 34.3M 9.6M 21.8M 6.7M
LinguisticEncoder 14M 3.4M
VariationalGenerator 11M 2.8M
FlowPostNet 9.3M 3.4M


DATASET refers to the names of datasets such as LJSpeech in the following documents.


You can install the Python dependencies with

pip3 install -r requirements.txt

Also, Dockerfile is provided for Docker users.


You have to download the pretrained models and put them in output/ckpt/DATASET/.

For a single-speaker TTS, run

python3 synthesize.py --text "YOUR_DESIRED_TEXT" --restore_step RESTORE_STEP --mode single --dataset DATASET

The generated utterances will be put in output/result/.

Batch Inference

Batch inference is also supported, try

python3 synthesize.py --source preprocessed_data/DATASET/val.txt --restore_step RESTORE_STEP --mode batch --dataset DATASET

to synthesize all utterances in preprocessed_data/DATASET/val.txt.


The speaking rate of the synthesized utterances can be controlled by specifying the desired duration ratios.
For example, one can increase the speaking rate by 20 by

python3 synthesize.py --text "YOUR_DESIRED_TEXT" --restore_step RESTORE_STEP --mode single --dataset DATASET --duration_control 0.8

Please note that the controllability is originated from FastSpeech2 and not a vital interest of PortaSpeech.



The supported datasets is

  • LJSpeech: a single-speaker English dataset consists of 13100 short audio clips of a female speaker reading passages from 7 non-fiction books, approximately 24 hours in total.



python3 prepare_align.py --dataset DATASET

for some preparations.

For the forced alignment, Montreal Forced Aligner (MFA) is used to obtain the alignments between the utterances and the phoneme sequences.
Pre-extracted alignments for the datasets are provided here.
You have to unzip the files in preprocessed_data/DATASET/TextGrid/. Alternately, you can run the aligner by yourself.

After that, run the preprocessing script by

python3 preprocess.py --dataset DATASET


Train your model with

python3 train.py --dataset DATASET

Useful options:

  • To use Automatic Mixed Precision, append --use_amp argument to the above command.
  • The trainer assumes single-node multi-GPU training. To use specific GPUs, specify CUDA_VISIBLE_DEVICES=<GPU_IDs> at the beginning of the above command.



tensorboard --logdir output/log

to serve TensorBoard on your localhost.


  • For vocoder, HiFi-GAN and MelGAN are supported.
  • Add convolution layer and residual layer in VariationalGenerator to match the shape of conditioner and output.
  • No ReLU activation and LayerNorm in VariationalGenerator for convergence of word-to-phoneme alignment of LinguisticEncoder.
  • Use absolute positional encoding in LinguisticEncoder instead of relative positional encoding.
  • Will be extended to a multi-speaker TTS.


Please cite this repository by the “Cite this repository” of About section (top right of the main page).