help deepspeed in oobadooga

[kalle07]

normal audio works fine, also with my own audiofiles
(win10, RTX4060, python 3.11)

iv installed deepspeed (windwos with atsetup.bat) while activated text-web-ui

thats happened if i press the "preview" button oc with a text line (all works with disabled deepspeed)

...
[AllTalk Model] DeepSpeed Activating
[AllTalk Model] Changing model (DeepSpeed can take 30 seconds to activate)
[AllTalk Model] Information If you have not set CUDA_HOME path, DeepSpeed may fail to load/activate
[AllTalk Model] Information DeepSpeed needs to find nvcc from the CUDA Toolkit. Please check your CUDA_HOME path is
[AllTalk Model] Information pointing to the correct location and use 'set CUDA_HOME=putyoutpathhere' (Windows) or
[AllTalk Model] Information 'export CUDA_HOME=putyoutpathhere' (Linux) within your Python Environment
[AllTalk Model] Unloading model
[AllTalk Model] XTTSv2 Local Loading xttsv2_2.0.2 into cuda
[2024-03-31 09:58:13,491] [INFO] [logging.py:96:log_dist] [Rank -1] DeepSpeed info: version=0.11.2+unknown, git-hash=unknown, git-branch=unknown
[2024-03-31 09:58:13,491] [WARNING] [config_utils.py:69:_process_deprecated_field] Config parameter replace_method is deprecated. This parameter is no longer needed, please remove from your call to DeepSpeed-inference
[2024-03-31 09:58:13,491] [WARNING] [config_utils.py:69:_process_deprecated_field] Config parameter mp_size is deprecated use tensor_parallel.tp_size instead
[2024-03-31 09:58:13,491] [INFO] [logging.py:96:log_dist] [Rank -1] quantize_bits = 8 mlp_extra_grouping = False, quantize_groups = 1
[AllTalk Model] Warning Model is still loading, please wait before trying to generate TTS
...

oc iv waited until it says "deepspeed enebled"

[erew123]

Hi @kalle07

It looks like you are running DeepSpeed 11.2 which was compiled for Pytorch 2.1.x

[2024-03-31 09:58:13,491] [INFO] [logging.py:96:log_dist] [Rank -1] DeepSpeed info: version=0.11.2+unknown, git-hash=unknown, git-branch=unknown

Text-gen-webui recently moved up to Pytorch 2.2.x, which requires a recompilation of DeepSpeed.

So you will need to uninstall DeepSpeed 11.2 and upgrade to the DeepSpeed 14.0 that I compiled to work on Pytorch 2.2.x, matching the correct version of CUDA.

I would suggest that if you are up to date with AllTalk, please load atsetup, run the diagnostics and on there you can find what version of Pytorch (torch) you are running as well as what CUDA version, which will let you know what version of DeepSpeed to install.

You should have an "Uninstall DeepSpeed" option on the menu for Text-gen-webui and also 4x installation options to match the correct version of Pytorch and DeepSpeed.

Thanks

[kalle07]

thx! it works now ;)

2-4 times faster ^^