An ASR server framework in Python, aiming to support both streaming and non-streaming recognition.

Note: Only non-streaming recognition is implemented at present. We will add streaming recognition later.

CPU-bound tasks, such as neural network computation, are implemented in C++; while IO-bound tasks, such as socket communication, are implemented in Python.

Caution: We assume the model is trained using pruned stateless RNN-T from icefall and it is from a directory like pruned_transducer_statelessX where X >=2.

We provide a Colab notebook, containing how to start the server, how to start the client, and how to decode test-clean of LibriSpeech.

Open In Colab


First, you have to install PyTorch and torchaudio. PyTorch 1.10 is known to work. Other versions may also work.

Second, clone this repository

git clone
cd sherpa
pip install -r ./requirements.txt

Third, install the C++ extension of sherpa. You can use one of the following methods.

Option 1: Use pip

pip install --verbose k2-sherpa


pip install --verbose git+

Option 2: Build from source with

python3 install

Option 3: Build from source with cmake

mkdir build
cd build
cmake ..
make -j
export PYTHONPATH=$PWD/../sherpa/python:$PWD/lib:$PYTHONPATH


First, check that sherpa has been installed successfully:

python3 -c "import sherpa; print(sherpa.__version__)"

It should print the version of sherpa.

Start the server

To start the server, you need to first generate two files:

  • (1) The torch script model file. You can use --jit=1 in pruned_transducer_statelessX from icefall.

  • (2) The BPE model file. You can find it in data/lang_bpe_XXX/bpe.model in icefall, where XXX is the number of BPE tokens used in the training.

With the above two files ready, you can start the server with the following command:

sherpa/bin/ \
  --port 6006 \
  --num-device 0 \
  --max-batch-size 10 \
  --max-wait-ms 5 \
  --feature-extractor-pool-size 5 \
  --nn-pool-size 1 \
  --nn-model-filename ./path/to/exp/ \
  --bpe-model-filename ./path/to/data/lang_bpe_500/bpe.model &

You can use ./sherpa/bin/ --help to view the help message.

We provide a pretrained model using the LibriSpeech dataset at

The following shows how to use the above pretrained model to start the server.

git lfs install
git clone

sherpa/bin/ \
  --port 6006 \
  --num-device 0 \
  --max-batch-size 10 \
  --max-wait-ms 5 \
  --feature-extractor-pool-size 5 \
  --nn-pool-size 1 \
  --nn-model-filename ./icefall-asr-librispeech-pruned-transducer-stateless3-2022-05-13/exp/ \
  --bpe-model-filename ./icefall-asr-librispeech-pruned-transducer-stateless3-2022-05-13/data/lang_bpe_500/bpe.model

Start the client

After starting the server, you can use the following command to start the client:

./sherpa/bin/ \
    --server-addr localhost \
    --server-port 6006 \
    /path/to/foo.wav \

You can use ./sherpa/bin/ --help to view the usage message.

The following shows how to use the client to send some test waves to the server for recognition.

sherpa/bin/ \
  --server-addr localhost \
  --server-port 6006 \
  icefall-asr-librispeech-pruned-transducer-stateless3-2022-05-13//test_wavs/1089-134686-0001.wav \
  icefall-asr-librispeech-pruned-transducer-stateless3-2022-05-13//test_wavs/1221-135766-0001.wav \

RTF test

We provide a demo ./sherpa/bin/ to decode the test-clean dataset from the LibriSpeech corpus.

It creates 50 connections to the server using websockets and sends audio files to the server for recognition.

At the end, it will display the RTF and the WER.

To give you an idea of the performance of the pretrained model, the Colab notebook Open In Colab shows the following results:

RTF: 0.0094
total_duration: 19452.481 seconds (5.40 hours)
processing time: 183.305 seconds (0.05 hours)
%WER = 2.06

Errors: 112 insertions, 93 deletions, 876 substitutions, over 52576 reference words (51607 correct)

If you have a GPU with a larger RAM (e.g., 32 GB), you can get an even lower RTF.


View Github