Add Matcha-TTS

[csukuangfj]

Note that it does not depend on lightning and it does not use *.yaml files for config.

TODOs

• Refactor the phonemizer part. Use piper-phonemizer
• Refactor the dataset part. Use lhotse for that
• Support exporting to onnx and deployment in sherpa-onnx.
• Specifically, we should support returning each token's timestamp and generated audio.
• Remove unused code and fix code style issues.

The commit
seems to be working. You can find the checkpoint epoch-140.pt at
https://huggingface.co/csukuangfj/tmp-files/tree/main/icefall-pr-1173/1

It can generate audible speech at epoch-140 with only 2 ODE steps

0.mov

1.mov

[csukuangfj]

Checkpoints and logs are available at
https://huggingface.co/csukuangfj/icefall-tts-ljspeech-matcha-en-2024-10-28

The following is a sample wave generated with the ONNX models exported from this PR.

python3 ./matcha/onnx_pretrained.py \
 --acoustic-model ./model-steps-6.onnx \
 --vocoder ./hifigan_v1.onnx \
 --tokens ./data/tokens.txt \
 --input-text "Ask not what your country can do for you; ask what you can do for your country." \
 --output-wav ./matcha-epoch-4000-step6-hfigian-v1.wav

matcha-epoch-4000-step6-hfigian-v1.mov

Please refer to the README.md of this PR for usage.

[JinZr]

noted! will look into it! Best Regards Jin

…

On Tue, Oct 29, 2024 at 12:43 Fangjun Kuang ***@***.***> wrote: @csukuangfj <https://github.com/csukuangfj> requested your review on: #1773 <#1773> Add Matcha-TTS. — Reply to this email directly, view it on GitHub <#1773 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOON42DFWEU3YMYBY7W2US3Z54G5ZAVCNFSM6AAAAABQBIMGJKVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJUHE3TIOJSHEYTGOI> . You are receiving this because your review was requested.Message ID: ***@***.***>

[JinZr]

LGTM 🚀