Project Page | arXiv |
Zhecheng Yuan*, Tianming Wei*, Shuiqi Cheng, Gu Zhang, Yuanpei Chen, Huazhe Xu
*The first two authors contribute equally.
conda env create -f environment.yml
The algorithms will use the Places dataset for data augmentation, which can be downloaded by running
wget http://data.csail.mit.edu/places/places365/places365standard_easyformat.tar
After downloading and extracting the data, add your dataset directory to the datasets list in cfgs/aug_config.cfg.
For training:
bash scripts/train.sh
For evaluation:
bash scripts/eval.sh ours
You should modify the model_path in mani_eval.py first. You would better to check the saved video. The recorded success rate might miss some successful trials. Meanwhile, you need to uncomment the get_termination function in the Python file for the tasks: ['xarm_close_dex', 'franka_dual_dex', 'franka_bowl_dex'] under envs/tasks/ to serve as evaluation metrics. However, it should remain commented out during training.
If you find our work useful, please consider citing:
@article{yuan2024learning,
title={Learning to Manipulate Anywhere: A Visual Generalizable Framework For Reinforcement Learning},
author={Yuan, Zhecheng and Wei, Tianming and Cheng, Shuiqi and Zhang, Gu and Chen, Yuanpei and Xu, Huazhe},
journal={arXiv preprint arXiv:2407.15815},
year={2024}
}
Our code is generally built upon DrQ-v2. The robot model built upon mujoco-menagerie . The website is borrowed from DP3. We thank all these authors for their nicely open sourced code and their great contributions to the community.
This repository is released under the MIT license.