Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tiny-cuda-nn and instant-ngp fail with memory error #5

Open
6ooflames opened this issue Mar 10, 2024 · 0 comments
Open

tiny-cuda-nn and instant-ngp fail with memory error #5

6ooflames opened this issue Mar 10, 2024 · 0 comments
Labels
implementation Unimplemented feature(s)

Comments

@6ooflames
Copy link

6ooflames commented Mar 10, 2024

I tried running instant-ngp and compiled it with nvcc with sm75, which had to be set manually since for some reason it defaulted to sm88, which isn't found in any nvidia gpu.
Sadly, there are certain memory allocation issues, and the program thinks it's running out of memory way before it touches all of the actual memory of my GPU. I recorded it peaking at 4/16GB utilization.
Using tiny-cuda-nn with the Fully Fused MLP, it says:
WARNING GPUMemoryArena: GPU 0 does not support virtual memory. Falling back to regular allocations, which will be larger and can cause occasional stutter. Uncaught exception: FullyFusedMLP: insufficient shared memory available on the GPU. Reduce "n_neurons" or use "CutlassMLP" (better compatibility but slower) instead.
With the Cutlass MLP, the error changes to:
Uncaught exception: /run/media/david/sda1/instant-ngp/dependencies/tiny-cuda-nn/include/tiny-cuda-nn/cutlass_matmul.h:330 status failed with error Error Internal

any input would be much appreciated.

cheers,
David

@lshqqytiger lshqqytiger added the implementation Unimplemented feature(s) label Apr 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
implementation Unimplemented feature(s)
Projects
None yet
Development

No branches or pull requests

2 participants