You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
RuntimeError: CUDA error: device-side assert triggered CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.
#12
Open
hgy123 opened this issue
Aug 12, 2023
· 0 comments
Hello!
when using the IPN-V2 network, I would like to obtain segmentation results for Largevessels and change the code from five class network to a two class network(the results of the five class network are normal). The following error occurred:
C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [832,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [480,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [481,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [929,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [930,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [33,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [800,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [804,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [805,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [704,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [705,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [706,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [903,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [904,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [329,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [330,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [129,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [130,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [131,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [232,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [233,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [234,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [104,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [105,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [207,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [208,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [683,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [684,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [685,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [780,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [781,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [1006,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [1007,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [555,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [556,0,0] Assertion
t >= 0 && t < n_classesfailed. C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\cuda\NLLLoss2d.cu:103: block: [0,0,0], thread: [6,0,0] Assertion
t >= 0 && t < n_classesfailed. Traceback (most recent call last): File "D:\download\Code\3D Baselines\train.py", line 161, in <module> train_net(net=net,device=device) File "D:\download\Code\3D Baselines\train.py", line 68, in train_net loss = Loss_CE(pred, train_annotations)+ 0.6 * Loss_DSC(pred, train_annotations) File "C:\Users\hgyy\AppData\Roaming\Python\Python39\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "D:\download\Code\3D Baselines\utils.py", line 226, in forward target=make_one_hot(target.long(),shape) File "D:\download\Code\3D Baselines\utils.py", line 160, in make_one_hot result.scatter_(1, input.cpu(), 1) RuntimeError: CUDA error: device-side assert triggered CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with
TORCH_USE_CUDA_DSAto enable device-side assertions.
My environment: python=3.9.16 torch=2.0.0 torchvision=0.15.1
Where did I make a mistake?
The text was updated successfully, but these errors were encountered: