Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

some training issue #151

Open
ZeKunZhang1998 opened this issue Jul 19, 2020 · 1 comment
Open

some training issue #151

ZeKunZhang1998 opened this issue Jul 19, 2020 · 1 comment

Comments

@ZeKunZhang1998
Copy link

Process 0: training loss at iteration 5: 30.145654678344727
Process 0: training loss at iteration 10: 20.740644454956055
0%| | 14/500000 [00:11<77:07:24, 1.80it/s]Traceback (most recent call last):
File "train.py", line 55, in prefetch_data
data, ind = sample_data(system_config, db, ind, data_aug=data_aug)
File "/content/drive/My Drive/CornerNet-Lite/core/sample/init.py", line 5, in data_sampling_func
return globals()[sys_configs.sampling_function](sys_configs, db, k_ind, data_aug, debug)
File "/content/drive/My Drive/CornerNet-Lite/core/sample/cornernet.py", line 135, in cornernet
draw_gaussian(tl_heatmaps[b_ind, category], [xtl, ytl], radius)
IndexError: index 6 is out of bounds for axis 1 with size 6
Process Process-3:
Traceback (most recent call last):
File "/content/anaconda3/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
self.run()
File "/content/anaconda3/lib/python3.7/multiprocessing/process.py", line 99, in run
self._target(*self._args, **self._kwargs)
File "train.py", line 59, in prefetch_data
raise e
File "train.py", line 55, in prefetch_data
data, ind = sample_data(system_config, db, ind, data_aug=data_aug)
File "/content/drive/My Drive/CornerNet-Lite/core/sample/init.py", line 5, in data_sampling_func
return globals()[sys_configs.sampling_function](sys_configs, db, k_ind, data_aug, debug)
File "/content/drive/My Drive/CornerNet-Lite/core/sample/cornernet.py", line 135, in cornernet
draw_gaussian(tl_heatmaps[b_ind, category], [xtl, ytl], radius)
IndexError: index 6 is out of bounds for axis 1 with size 6
Process 0: training loss at iteration 15: 18.72808074951172
0%| | 15/500000 [00:12<77:24:29, 1.79it/s]Exception in thread Thread-1:
Traceback (most recent call last):
File "/content/anaconda3/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/content/anaconda3/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "train.py", line 68, in pin_memory
data = data_queue.get()
File "/content/anaconda3/lib/python3.7/multiprocessing/queues.py", line 113, in get
return _ForkingPickler.loads(res)
File "/content/anaconda3/lib/python3.7/site-packages/torch/multiprocessing/reductions.py", line 256, in rebuild_storage_fd
fd = df.detach()
File "/content/anaconda3/lib/python3.7/multiprocessing/resource_sharer.py", line 57, in detach
with _resource_sharer.get_connection(self._id) as conn:
File "/content/anaconda3/lib/python3.7/multiprocessing/resource_sharer.py", line 87, in get_connection
c = Client(address, authkey=process.current_process().authkey)
File "/content/anaconda3/lib/python3.7/multiprocessing/connection.py", line 492, in Client
c = SocketClient(address)
File "/content/anaconda3/lib/python3.7/multiprocessing/connection.py", line 619, in SocketClient
s.connect(address)
FileNotFoundError: [Errno 2] No such file or directory

Process 0: training loss at iteration 20: 14.347289085388184
0%| | 21/500000 [00:15<63:23:04, 2.19it/s]

help me, please

@alaa-shubbak
Copy link

did you solve this error , any help please?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants