Min Memory requirement?

[shersoni610]

Hello,

I am unable to run the code on GTX 4070 with 12GB memory. I get the following error:

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 1024.00 MiB. GPU 0 has a total capacity of 11.73 GiB of which 285.94 MiB is free. Process 4646 has 298.61 MiB memory in use. Including non-PyTorch memory, this process has 10.81 GiB memory in use. Of the allocated memory 8.91 GiB is allocated by PyTorch, and 1.67 GiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)

Is there a way to reduce the memory requirement?

Thanks.

[staple-pi]

maybe you can try smaller model_type such as 'vit_l'

[cpuhrsch]

What batch size did you use?

[realVegetable]

I haved encountered the same problem.
I use rtx 2080 super.
But I used to run the original segment anything successfully on my rtx 2080 super.

[cpuhrsch]

@realVegetable - What batch size and model do you use?

[realVegetable]

@cpuhrsch - all models run well including "vit_h" "vit_l" "vit_b" . I just use the "SamAutomaticMaskGenerator" , and I don't change any params except the model, so the batch size is default.

[cpuhrsch]

@realVegetable - Can you give the following snippet a shot
https://github.com/pytorch-labs/segment-anything-fast/blob/main/amg_example/amg_example.py#L42-L44

sam = sam_model_fast_registry[model_type](checkpoint=sam_checkpoint)
sam.to(device=device)
mask_generator = SamAutomaticMaskGenerator(sam, process_batch_size=8)

In particular you can adjust process_batch_size to attenuate the amount of memory. A smaller value means less memory consumption (but potentially worse performance).

Thank you

[realVegetable]

@cpuhrsch - It didn't work. Using vit_h model and vit_b model, I set process_batch_size=8, but still failed with OutOfMemoryError.

OutOfMemoryError: CUDA out of memory. Tried to allocate 3.52 GiB. GPU 0 has a total capacity of 8.00 GiB of which 0 bytes is free. Of the allocated memory 16.51 GiB is allocated by PyTorch, and 3.55 GiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)

[cpuhrsch]

Hm, can you try process_batch_size=1 @realVegetable ?