Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ti.ad.no_grad decorator and needs_grad=False not prevent creating gradients tensors #8460

Open
byphilipp opened this issue Jan 10, 2024 · 0 comments

Comments

@byphilipp
Copy link

byphilipp commented Jan 10, 2024

Describe the bug
I using the function with ti.ad.no_grad decorator but taichi engine creates gradients tensor (on forward pass!!)

To Reproduce

import torch
device = 'cuda'
import taichi as ti
ti.init(getattr(ti,device))

ti2d = ti.types.ndarray(ndim=2, needs_grad=False)

@ti.kernel
def taichi_sinfactor (x: ti2d, out: ti2d, factor: ti.f32):
    for i,j in x:
        out[i,j] = ti.sin(x[i,j])*factor

@ti.ad.no_grad
def taichi_nograd(x, out, factor):
    taichi_sinfactor (x, out, factor)

N = 64
x = torch.rand([N,N],device=device,requires_grad=True)
factor = 5.
out = torch.zeros_like(x)
print([x.grad, out.grad])
taichi_nograd(x, out, factor)
print([x.grad, out.grad])

Log/Screenshots

$ python my_sample_code.py
[Taichi] version 1.8.0, llvm 15.0.1, commit 5a994612, win, python 3.8.8

[I 01/10/24 11:50:00.948 13700] [shell.py:_shell_pop_print@23] Graphical python shell detected, using wrapped sys.stdout
[Taichi] Starting on arch=cuda
[None, None]
[tensor([[0., 0., 0.,  ..., 0., 0., 0.],
        [0., 0., 0.,  ..., 0., 0., 0.],
        [0., 0., 0.,  ..., 0., 0., 0.],
        ...,
        [0., 0., 0.,  ..., 0., 0., 0.],
        [0., 0., 0.,  ..., 0., 0., 0.],
        [0., 0., 0.,  ..., 0., 0., 0.]], device='cuda:0'), None]

Additional comments

The hotfix

def taichi_nograd(*argin):
    return taichi_sinfactor (*[a.detach() if isinstance(a,torch.Tensor) else a for a in argin])
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Untriaged
Development

No branches or pull requests

1 participant