Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable integration with Metal API for accelerated generation on Apple Silicon #227

Open
AriaShishegaran opened this issue Mar 28, 2024 · 2 comments

Comments

@AriaShishegaran
Copy link

Hi, thanks for the fantastic job. Do you plan to add integration with Apple Silicon, right now it is actively looking for CUDA cores and fails to do so, therefore, sticking with the CPU.

@AriaShishegaran AriaShishegaran changed the title Enable integration with Metal APIs for accelerated generation on Apple Silicon Enable integration with Metal API for accelerated generation on Apple Silicon Mar 28, 2024
@aedocw
Copy link
Owner

aedocw commented Mar 28, 2024

There's a PR for that available here:
#181

Prior to that PR, I had done some testing and found it made no improvement. I think it will be useful if/when there's basically full CUDA compatibility and it work with deepspeed. I have not checked for progress in the last few weeks, but when I did last check I saw there were still quite a few missing operators that XTTS needs.

@AriaShishegaran
Copy link
Author

Got it. is this, btw, something that needs to be addressed on Pytorch's side for it to fully leverage Metal API?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants