I get error message when using Finetuning

[TomiTom1234]

I wanted to train a voice, I did put a 6 min sample wav audio file in the ..alltalk_tts\finetune\put-voice-samples-in-here folder, then started the start_finetune.bat, followed the first step, it created tmp-trn folder with metadata_eval.csv and metadata_train.csv files in it.

But when I tried to start step 2, the following error messages came up in the command prompt:
`[FINETUNE] Starting finetuning on Base Model

DVAE weights restored from: E:\AI\alltalk_tts\models\xttsv2_2.0.2\dvae.pth
Traceback (most recent call last):
File "E:\AI\alltalk_tts\finetune.py", line 1466, in train_model
config_path, original_xtts_checkpoint, vocab_file, exp_path, speaker_wav = train_gpt(language, num_epochs, batch_size, grad_acumm, train_csv, eval_csv, output_path=str(output_path), max_audio_length=max_audio_length)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\AI\alltalk_tts\finetune.py", line 670, in train_gpt
train_samples, eval_samples = load_tts_samples(
^^^^^^^^^^^^^^^^^
File "E:\AI\alltalk_tts\alltalk_environment\env\Lib\site-packages\TTS\tts\datasets_init_.py", line 121, in load_tts_samples
assert len(meta_data_train) > 0, f" [!] No training samples found in {root_path}/{meta_file_train}"
^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: [!] No training samples found in E:\AI\alltalk_tts\finetune\tmp-trn/E:\AI\alltalk_tts\finetune\tmp-trn\metadata_train.csv`

Any idea what is going on and how to solve it?

[erew123]

Hi @TomiTom1234

Have you manually changed the paths on step 2 to the files? These fields would normally be auto populated. I ask because in the error above, the path to the file refers to the E drive twice:

No training samples found in E:\AI\alltalk_tts\finetune\tmp-trn/E:\AI\alltalk_tts\finetune\tmp-trn\metadata_train.csv`

and I would assume the correct path is:

E:\AI\alltalk_tts\finetune\tmp-trn\metadata_train.csv

So that looks a bit odd to me, hence wondering if you changed anything?

Otherwise, its a bit hard to tell what occurred as I cannot see the console screen for step 1. However in the console it should tell you what file it found and how much audio it found:

So you may wish to take a look at that and see if your audio was processed correctly.

Beyond that I ran through the current version of finetuning just now and can confirm its working ok. Here are my screenshots:

On step one, in the logs and the console it tells you the files and the path it created

---

---

And then step 2, the fields should auto populate, with the files specified in step 1, and would match the locations specified in the terminal.

If you have built your own training data set you can specify your own files here, but the path has to be correct.

---

---

I did run a training run on the current build of Finetuning (as above "Model training done. Move to Step 3"), so I cant see a problem with the code. So it either has to be:

1. Your audio files not being able to be read/processed, so look at the step 1 logs.
2. The paths in Step 2 are not pointing at the correct files.

Thanks

[TomiTom1234]

Thanks for taking the time to respond.
No I didn't change anything, except the language to match the spoken language of the audio file. Here is the screenshot after the first step:

And here after the step 2 and the error message in the command prompt:

And here how it looks and reads inside the console itself:

I didn't change anything, all by default (except the language from EN to AR).

[erew123]

Hi @TomiTom1234

I think I have found your problem.

Whisper, which is used to break down the original sample to individual samples and then create the training CSV files, seems to be pretty poor at working with Arabic. I am concluding this from looking at the Whisper model information here and also people generally saying its not great based on this 2 year old conversation here

So down to the technical of what Step 1 is/isnt doing. In short, its not picking out the spoken language correctly and creating your training data. It should be populating the wavs folder with wav files and also populating the training csv files.

You could see if the Whisper Large-v3 is any better at this. It may be, I cannot say.

---

---

The only other alternative would be to manually make your own training data set. I am working on making this process simpler, but I haven't gotten there yet. The way you do this however is:

1. Split your main audio down into to small samples of spoken speech. So your 6 minute file would need to be broken into multiple smaller files.

2. Create a folder structure as follows (you can always train Step 1 in en to see how a dataset/folder structure is built):

finetune/
│
└── tmp-trn/
    │
    │── metadata_eval.csv
    ├── metadata_train.csv
    ├── lang.txt
    │
    └── wavs/

3. In the wavs folder is where you would put your sample wav files.

4. The lang.txt file literally would have ar written in it.

5. As this is Coqui's scripts performing the training, its based on their training setup/format, detailed here

You would have to create your metadata_train.csv and metadata_eval.csv files. Typically, metadata_eval.csv would only reference a list of 15% of your sample wav files of the total 100% of wav samples you have. You should not repeat wav file names in the two CSV files.

The 1st line of each csv file should contain audio_file|text|speaker_name and this also tells you the format you use. Basically there is a pipe | between each section.

So it would be:

wavfilenumer001.wav|this is what is spoken in the wav file| coqui
wavfilenumer002.wav|this is the transcribed text for this wav file| coqui
etc

6. Once you have completed this, with the files in the correct location, you can immediately go to Step 2, populate the correct locations of your files and run the training:

If you need more information on building training data and datasets, I would suggest looking around the Coqui documentation and discussions area.

Thanks