MyoChallenge '24 Annoucements Thread

[cherylwang20]

Hi, MyoChallenge '24 participants and MyoSuite community,

In the past two months, we have received some questions in regard to how the observation space can be used and how freely can new observations be added. We would like to take this chance to fully clarify it.

During Training:
Participants are free to modify the observations function directly in env.get_obs_dict() to include additional observations that your policy requires for training. We recommend that participants keep the standard observations dictionary untouched, and add your custom observations by working with the data from the existing observation dictionary. Be sure to modify the observation keys (DEFAULT_OBS_KEYS) and pass in a list of observations you would want to use.

During Evaluation:
In this stage, participants will not be able to change the observation dictionary in the env.get_obs_dict() (get_obs_dict function in run_track_v0.py and bimanual_v0.py). Instead, participants will have full access to the standard observation dictionary but will only be able to define custom observations from the standard observation dictionary (hence the recommendation to postprocess your custom observations from the standard observation dictionary during training). Tutorials for submissions will be released as soon as they are ready and announced via our social media accounts.

Hope this clarifies any further concerns and feel free to use this thread to discuss more.

[XiaobenLi00]

Hi,

Thanks for your clarifies! So does it mean that during training I can use my custom env variants (with modified obs_dict, rwd_dict, etc...), but for evaluation I must use the original env without any modifications?

[P-Schumacher]

Yes! You can use anything you want for training.

for evaluation, you are restricted to the quantities provided in the obs_dict, but you can construct any kind of observation you want from them.

otherwise the evaluation environment is fixed.

[cherylwang20]

#NeurIPS #MyoChallenge 2024 Submissions are LIVE!!!

MyoChallengers, you can now register your team on EvalAI, submit your best solution and start competing against teams from everywhere in the world 🌎 Follow the link here to start the Challenge: https://lnkd.in/dFpURga7 !

1. Register a team on EvalAI & participate in MyoChallenge 2024! https://eval.ai/web/challenges/challenge-page/2373/overview
2. Clone our public template for submissions: https://github.com/MyoHub/myochallenge_2024eval
3. Checkout the instructions and submit a testing solution by following the Github/DIY submission.

You can also ask for Google Cloud Credits for computing, just send an email to [email protected] with your Team name.

[cherylwang20]

🚀 MyoChallenge '24 Leaderboard is now LIVE! 🚀

Curious about the new Phase 2 variations? Need guidance on EvalAI submissions? Got burning questions for the organizers?
Join us at the first MyoChallenge '24 Workshop & Q&A on Sep 30th, 2024 at 10:00 AM EST.

Register now 👇
https://forms.gle/tLHJqsNGWpJ9sNCx6

[cherylwang20]

🚀 Announcement: MyoChallenge 2024 - MyoSuite v2.8.2 Bugfix Release for Locomotion Track.

🔧 New in v2.8.2: Critical fixes for locomotion observations, including joint angles and heightfields. To Ensure top performance: Update to the latest MyoSuite version before submitting your MyoChallenge entries.

[cherylwang20]

!ATTENTION MyoChallenge Participants!
🚀 Announcement: MyoChallenge 2024 - MyoSuite v2.8.3 Bugfix Release for Manipulation Track.
🔧 New in v2.8.3: Critical fixes for evaluation threshold for solved condition.

To Ensure top performance: Update to the latest MyoSuite version before submitting your MyoChallenge entries.