Conqui v2 2.0.3 sounding better somehow.

[311-code]

A bit different than erew123's chat findings from back in November here, and not sure why. I just tried to train over the coqui v2 2.0.3 model and got less "feeling they are reading from a script" when voice talks.

Accuracy similar or better and better flow. I'd recommend trying it out again with method below just to make sure, link here: https://huggingface.co/coqui/XTTS-v2/tree/v2.0.3

For anyone here that wants to try: Backup the files in \extensions\alltalk_tts\models\xttsv2_2.0.2 download all the new files and put in there. Train over base again.

Not sure if this was the difference: I merged 3 of the most accurate samples from the previous trainings in voices folder. Merged them into one file as a .wav and placed in the \extensions\alltalk_tts\finetune\put-voice-samples-in-here folder, and used that for finetuning (in addition to the original dataset) then used that same merged .wav file for the voice selection.

[erew123]

Hi @brentjohnston

Ill move this over to the Discussions area.

With the 2.0.3 model, when they first released it, my god did it sound terrible coqui-ai/TTS#3306

I'm not sure if they messed something up with the models configuration file. However. I believe they did eventually correct this....eventually.

Id welcome people to play with it and give feedback. If enough people give feedback that its overall better, I can swap the base model on initial setup of AllTalk.

You can actually download the \extensions\alltalk_tts\models\xttsv2_2.0.3 folder and change the path in the settings page.

If people want to test out and feed back, Im happy to listen.

Thanks

[311-code]

Just wanted to add these personal findings here for anyone reading also, let me know your results:

If you were just having it autosegment before and want to get more detailed with this, during Part 1 of the finetuning app go to \extensions\alltalk_tts\finetune\tmp-trn\wav, delete all the wav files whisper just generated, and make your own. (you'll also get two .csv files here)

The new wavs you make should be all be one or two sentences each and not exceed 250 characters per sample. Then edit the two .csv files in \finetune\ with the exact words that are being spoken for each .wav file. Use 22050hz or so for the wavs. I did about 74 samples in total in my training.

I typed long words occasionally like "oooh" instead of simply "oh" on some of them if it sounded like I really stretched it out long, and and it seemed to work (not sure if this is right though). I got very detailed with it and typed when I said "I I don't know" if I said "I" twice. This is basically the equivelant of Whisper v6 I think ;) Then started the training. This way of doing it manually still seems better, but if you want something fast and still good whisper v2 is totally fine.

Then when training is done, on the last training page, copy the model to the training folder (ignore the page that only shows you a few samples to test it)

Copy the samples you made to voices folder. In oogabooga take notes of every sample, and which ones sound the most like you over the trained model.

Once you find the best ones. Let's say it's 5 of them that sound exactly like you but a little different. Combine them into one single sample and put that file in \extensions\alltalk_tts\voices. Select "XTTSv2 FT" button again (the new finetuned model, and use that single wav for your voice sample. It should now sound much better, more, accurate, and less instances of a robotic flow. It also helps to not use .wavs of you reading stuff from a script and instead just naturally speaking. For some reason I also got the best results on this specific finetune using temperature 0.95 and 4.5 repeat penalty slider settings.