Ant-Dir Environment

[Folly135]

Hi! When I generate the offline dataset corresponding to Ant-dir, I can only generate the data of goal1, and the data in the goal0 folder is always empty, how should I solve this problem?
Moreover, in the offline dataset corresponding to the cheetah-vel I generated, there is no data with train epoch 6e5, I changed the config to null in order to run it successfully, is this change correct?

[LanqingLi1993]

@Folly135 Hi, I have never encountered the first problem, can you double-check if n_tasks in https://github.com/LanqingLi1993/FOCAL-ICLR/blob/master/configs/ant-dir.json is 2 and try to delete the goal0 folder to start over again?

For the second problem, "null" means you will use all data in the directory to train your models. If that's what you want, then it's correct.

If you find the repo and answer helpful, please star it, thank you. =)

[Folly135]

@LanqingLi1993 Hi! I saw the complete dataset zip you shared and compared it to the one I generated which does have issues, thanks for sharing. I would also like to ask if I can delete the npy data files for the rest of the epochs when the train epoch and eval epoch set in config are not null, it doesn't seem to affect loading the corresponding data.

[LanqingLi1993]

@Folly135 Hi, if you specify an epoch number rather than null in the train epoch and eval epoch, the program will load all trajectories corredponding to that epoch for training/testing. If you are certain that we will never use other npy data files, it's fine to delete them.

Just be aware that the data loading utilities (e.g. init_buffer() in https://github.com/LanqingLi1993/FOCAL-ICLR/blob/master/rlkit/core/rl_algorithm.py) were only designed for my own workflow and certainly not perfect. Feel free to re-invent them for your own projects. =)

[Folly135]

@LanqingLi1993 Hi, I met another problem when I ran the ant-dir environment.
ValueError: Cannot take a larger sample than population when 'replace=False'.
I found that adding "mb_replace": 1 to the config file in ant-dir.json works fine, but I'm not sure if this is the right change.
Moreover, the same problem occurs when using the cheetah-dir environment.

[Folly135]

@LanqingLi1993 Another issue is that I can't successfully reproduce the results in Figure 3 inside the paper in both cheetah-vel and walker_random_params environments with the offline dataset you provided.
I don't know if I'm having a problem with the execution of my program somewhere, but the program does run smoothly and reproduces similar results in both the Point-Robot-Wind and Sparse-Point-Robot environments.
In addition, the program does not run successfully in Ant-Dir and Cheetanh-Dir environments.
I hope to hear from you about the solutions to these problems, and I would like to thank you for your help.

[LanqingLi1993]

@Folly135 FOCAL needs to sample a batch of tasks (meta_batch) to compute the metric loss. replace=False means it cannot sample duplicate task indices, therefore meta_batch cannot be greater than the population (i.e., n_train_tasks or n_eval_tasks). Indeed you should set mb_replace=1 which means duplicate tasks can be sampled, otherwise there won't be positive pairs (latent z from the same task) in the metric loss. I have revised the mb_replace in https://github.com/LanqingLi1993/FOCAL-ICLR/blob/master/configs/default.py to reflect this change.

For the reproduction issue, we did receive several reports and will provide an updated version of codebase at https://github.com/LanqingLi1993/FOCAL-latest, please check it out in a few days. Thank you for following our work!

[Folly135]

@LanqingLi1993 Thank you very much for your reply!
I checked the Ant-Dir and Cheetanh-Dir environments that I previously executed with mb_replace=1 modified, and their corresponding results are also very different from the results in the paper.
Will successful reproduction in these two environments be covered in the new shared link?
Thank you very much! : )

[LanqingLi1993]

@Folly135 You are welcome! Yes, you should be able to successfully reproduce Ant and HalfCheetah Envs using our latest code: https://github.com/LanqingLi1993/FOCAL-latest. Please let us know if you encounter any further issues. Thank you!