Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support vits models from piper #390

Merged
merged 1 commit into from
Oct 26, 2023

Conversation

csukuangfj
Copy link
Collaborator

Usage

I have converted the model en_US-lessac-medium from piper to sherpa-onnx
and you can find the converted model in the following repo:
https://huggingface.co/csukuangfj/vits-piper-en_US-lessac-medium

If you are interested in how the model is converted, please have a look at the following colab notebook:
https://colab.research.google.com/drive/1PScLJV3sbUUAOiptLO7Ixlzh9XnWWoYZ?usp=sharing

In the following, we describe how to use the converted model with sherpa-onnx

Download models

wget https://huggingface.co/csukuangfj/vits-piper-en_US-lessac-medium/resolve/main/en_US-lessac-medium.onnx

wget https://huggingface.co/csukuangfj/vits-piper-en_US-lessac-medium/resolve/main/lexicon.txt

wget https://huggingface.co/csukuangfj/vits-piper-en_US-lessac-medium/resolve/main/tokens.txt

Build sherpa-onnx

Please use the code from this PR and build sherpa-onnx from source by following
https://k2-fsa.github.io/sherpa/onnx/install/index.html

After building sherpa-onnx, you can use

./build/bin/sherpa-onnx-offline-tts \
  --output-filename=./piper.wav \
  --vits-length-scale=0.8 \
  --vits-model=./vits-piper-en_US-lessac-medium/en_US-lessac-medium.onnx \
  --debug=1 \
  --vits-lexicon=./vits-piper-en_US-lessac-medium/lexicon.txt \
  --vits-tokens=./vits-piper-en_US-lessac-medium/tokens.txt \
  "Liliana, the most beautiful and lovely assistant of our team!"

It will generate a wave file ./piper.wav. I have converted it to mov and attached it below so that you can listen to it.

piper.mov

Your help is needed

Since piper provides a lot of pre-trained onnx models, it would be great if you can help convert them to sherpa-onnx.

Unlike piper, we don't need to install piper_phonemize. We have converted all the words into lexicon.txt.

@csukuangfj csukuangfj merged commit 4451285 into k2-fsa:master Oct 26, 2023
42 of 141 checks passed
@csukuangfj csukuangfj deleted the support-vits-piper branch October 26, 2023 06:10
@csukuangfj
Copy link
Collaborator Author

Note that you can find all the pre-trained ONNX models from piper at the following address
https://huggingface.co/rhasspy/piper-voices/tree/main

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant