Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error when training model #399

Closed
M-S-10 opened this issue Feb 29, 2024 · 2 comments
Closed

Error when training model #399

M-S-10 opened this issue Feb 29, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@M-S-10
Copy link
Contributor

M-S-10 commented Feb 29, 2024

Describe the bug
Failing to convert data to tfrecord

To Reproduce
Steps to reproduce the behavior:

  1. Uploaded the sessions
  2. Divided into training and testing
  3. Started model training

Expected behavior
Initialization of model training

Screenshots
Screenshot 2024-02-29 125843
Screenshot 2024-02-29 130052

Desktop (please complete the following information):

  • OS: Windows 11
  • Browser : Any

Additional context
conda_output_log.txt

Oops! Image D:/T-Works cannot be found.
TFRecord file created successfully.
broadcast message Loading data from tfrecord...
Number of training instances: 0
Number of test instances: 0
broadcast failed
Exception in callback _chain_future.._set_state(<Future pendi...tures.py:384]>, <Future at 0x...StopIteration>) at D:\mssha\Apps\Anaconda\envs\openbot\lib\asyncio\futures.py:378
handle: <Handle _chain_future.._set_state(<Future pendi...tures.py:384]>, <Future at 0x...StopIteration>) at D:\mssha\Apps\Anaconda\envs\openbot\lib\asyncio\futures.py:378>
Traceback (most recent call last):
File "D:\mssha\Apps\Anaconda\envs\openbot\lib\asyncio\events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File "D:\mssha\Apps\Anaconda\envs\openbot\lib\asyncio\futures.py", line 380, in _set_state
_copy_future_state(other, future)
File "D:\mssha\Apps\Anaconda\envs\openbot\lib\asyncio\futures.py", line 356, in _copy_future_state
dest.set_exception(_convert_future_exc(exception))
TypeError: StopIteration interacts badly with generators and cannot be raised into a Future

@M-S-10 M-S-10 added the bug Something isn't working label Feb 29, 2024
@M-S-10
Copy link
Contributor Author

M-S-10 commented Mar 3, 2024

Running in C Drive now. TF Record files are getting created but another error at the next step.

@M-S-10
Copy link
Contributor Author

M-S-10 commented Mar 3, 2024

TFRecord file created successfully.
broadcast message Loading data from tfrecord...
Image shape: (96, 256, 3)
Command: 0.0
Label: [0.23529412 0.23529412]
Number of training instances: 28238
Number of test instances: 8671
broadcast preview None
broadcast failed [Errno 2] No such file or directory: 'C:\Users\mssha\OpenBot\policy\models\openbot_cil_mobile_lr0.0003_bz128_bn\model.png'
Future exception was never retrieved
future: <Future finished exception=FileNotFoundError(2, 'No such file or directory')>
Traceback (most recent call last):
File "D:\Anaconda\envs\openbot\lib\concurrent\futures\thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "C:\Users\mssha\OpenBot\policy\openbot\server\api.py", line 211, in train
raise e
File "C:\Users\mssha\OpenBot\policy\openbot\server\api.py", line 205, in train
tr = start_train(params, my_callback)
File "C:\Users\mssha\OpenBot\policy\openbot\train.py", line 624, in start_train
do_training(tr, callback, verbose)
File "C:\Users\mssha\OpenBot\policy\openbot\train.py", line 437, in do_training
tf.keras.utils.plot_model(model, to_file=dot_img_file, show_shapes=True)
File "D:\Anaconda\envs\openbot\lib\site-packages\keras\utils\vis_utils.py", line 440, in plot_model
dot.write(to_file, format=extension)
File "D:\Anaconda\envs\openbot\lib\site-packages\pydot\core.py", line 1663, in write
with io.open(path, mode="wb") as f:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\Users\mssha\OpenBot\policy\models\openbot_cil_mobile_lr0.0003_bz128_bn\model.png'
Unregistering...
unhandled exception during asyncio.run() shutdown
task: <Task finished name='Task-3' coro=<_run_app() done, defined at D:\Anaconda\envs\openbot\lib\site-packages\aiohttp\web.py:287> exception=NameError("name 'info' is not defined")>
Traceback (most recent call last):
File "D:\Anaconda\envs\openbot\lib\site-packages\aiohttp\web.py", line 429, in _run_app
await asyncio.sleep(delay)
File "D:\Anaconda\envs\openbot\lib\asyncio\tasks.py", line 652, in sleep
return await future
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:\Anaconda\envs\openbot\lib\site-packages\aiohttp\web.py", line 431, in _run_app
await runner.cleanup()
File "D:\Anaconda\envs\openbot\lib\site-packages\aiohttp\web_runner.py", line 293, in cleanup
await site.stop()
File "D:\Anaconda\envs\openbot\lib\site-packages\aiohttp\web_runner.py", line 76, in stop
await self._runner.shutdown()
File "D:\Anaconda\envs\openbot\lib\site-packages\aiohttp\web_runner.py", line 369, in shutdown
await self._app.shutdown()
File "D:\Anaconda\envs\openbot\lib\site-packages\aiohttp\web_app.py", line 423, in shutdown
await self.on_shutdown.send(self)
File "D:\Anaconda\envs\openbot\lib\site-packages\aiohttp\signals.py", line 34, in send
await receiver(*args, **kwargs) # type: ignore
File "C:\Users\mssha\OpenBot\policy\openbot\server\zeroconf.py", line 70, in on_shutdown
await do_close(zc)
File "C:\Users\mssha\OpenBot\policy\openbot\server\zeroconf.py", line 64, in do_close
await zc.unregister_service(info)
NameError: name 'info' is not defined

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant