Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixing errors with _obtain_steps function #27

Open
alexofficial opened this issue Feb 16, 2024 · 1 comment
Open

Fixing errors with _obtain_steps function #27

alexofficial opened this issue Feb 16, 2024 · 1 comment

Comments

@alexofficial
Copy link

Hi,

I tried to collect the offline dataset of an online trained model using the _obtain_steps function. I found an error there, in lines 694-697, where two parenthesis are missing inside the file: scope-rl/scope-rl/dataset/synthetic.py
image

The solution is:

actions = np.zeros((n_trajectories * step_per_trajectory, self.action_dim))
action_probs = np.zeros((n_trajectories * step_per_trajectory,self.action_dim))

Secondly, there is a variable rollout_lengths (line 704), which causes an error in line 723 within the for loop. However, since there are no examples with _obtain_steps, I could not compare this. A quick solution was to simply change it from:
for rollout_step in rollout_lengths[i]:to forrollout_step in range(rollout_lengths[i]):`

Note: I have now successfully trained an offline RL model with the _obtain_steps function. Let me know if I can contribute in any way :)

@aiueola
Copy link
Collaborator

aiueola commented Feb 27, 2024

@alexofficial

Thank you for reaching out with the issues and solutions! The codes (both lines 694-697 and line 723) should definitely be fixed as you mentioned. Could you make a PR with the suggested update? We appreciate your help in improving the software.

alexofficial pushed a commit to alexofficial/scope-rl that referenced this issue Mar 6, 2024
The error occurred when the for loop was used with `rollout_lengths[i]`,
where `rollout_lengths` was created with `self.random_.choice()`,
resulting in a `numpy.int32` object.

The problem has been fixed by iterating directly over the range of `rollout_lengths[i]` instead.

Fixing errors with _obtain_steps function hakuhodo-technologies#27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants