Various questions about implementation #239
Unanswered
cestpasphoto
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi,
First, thank you very much for all contributors to this repo which is highly pedagogical!
As a newbie in reinforcement learning and machine learning in general, I started to fork your project and during this, it raised some questions to me:
Random generation of batch
In NNet.py, samples for each batch are randomly sampled independently of previous batches of same epoch. Pytorch and Tensorflow implementations share same technic.
But in this stackoverflow answer, training code is different and ensure no common sample between batches inside same epoch. Are both approaches acceptable?
Double random?
In Coach.py,
trainExamples
is shuffled before callingnnet.train(trainExamples)
. But what is the point if batches are sampled completely randomly from it?No use of lr variable?
It looks like learning rate parameter
lr
isn't used at all, especially not in Adam optimizer constructor, same in tensorflow implementation. Did I missed something?Optimizer history
In NNet.py, Adam optimizer is re-created at each iteration. Wouldn't it be better to keep it and reuse it? Maybe the optimizer has some kind of history and it's better not to start from scratch at each training?
Thank you in advance!
Beta Was this translation helpful? Give feedback.
All reactions