October 2024 Planning Meeting

[SimoneMic]

Update meeting about the current work in progres and the short-term goals for the current month

Chair:

• @SimoneMic

Attendees:

• @vigisushrutha23
• @steb6
• @andrearosasco

Absent:

• @PasMarra

---

@andrearosasco

September

• Behavior Cloning for ICRA paper: working on finalizing simulation experiments and porting everything on PANDA robot (for further validation).
• After ICRA deadline, the goal is to port the work on ergoCub.

October

• Submitted Paper, if rejected: IROS, journal otherwise.

TODO

• Improve paper result
• Do experiments on ergoCub: taking dataset with metaCub (improve demo on coffe machine with markers to improve reilability and teleoperation)
• Help on metaCub

@steb6

September

• Metacub working on simulation: debugged using ROS2 and ported the left-handed reference frames from Meta to the proper right-handed of the robot (thanks also to @PasMarra). The performance issue was resolved by improving the cartesian controller (see Pasquale's side).
• For this month there will be more focus on research: the idea comes from this paper and Stefano is working on solving a bug form the authors' code, that prevents from replicating the paper's results.
• Improve the current few-shot action recognition pipeline with CLIP embeddings, to add the possibility to interact with objects.
• After the ICRA deadline is due, we will test Metacub teleoperation on the real robot (we don't want to break it too often before the deadline).

October

• MetaCub issues resolved: tested on the real robot. Fixed the camera communication with the robot and implemented threading. The neck controller is really smooth. Issue when teleoperating arms, due to comunication rpc (maybe?), need to investigate further.

TODO

• Investigate (and solve) issue with slow/laggy arms teleoperation.
• Need to show depth pointclound on VR (WIP)
• (Research) Testing a way to creating fine-grained episodic tasks for few-shot-learning (see workbook)
• Festival della Scienza

@vigisushrutha23

September

• Working on ICRA paper submission: the human avoidance module is working, and its reactiveness has been improved. However, a bug on the bimanual manipulation module is accumulating the tracking error, and the robot stops and reaches a singularity. Currently working to solve this bug and then collect the final experiment data.
• After the deadline, will resume work on MPC

October

• ICRA paper submitted

TODO

• Working on demo integration for bimanual manipulation.
• Working on MPC, replaced Johns QP solver with std library, working on time-horizon.
• Bimanual improvement: use reinforcement learning with Federico's work for reliable grasping.

@PasMarra

Absent

• Worked on Metacub and improved the cartesian controller: now it also accepts the desired velocity as imput, improving thus the reactiveness of the controller. See these issues for further infos: about reference frames and test
• Currently working on tactile sensors characterization experiments setup: waiting for the 3D printing of a set of indenters. Afterwards an experimental campaign will be held to acquire data. The software setup is ready.
• Will start working on the sensor characterization with the data acquired.

@SimoneMic

September

• Finalized VLMaps with raycasting and improved resources utilization: reduced the amount of VRAM needed.
• Opened issue for designing the weights support for ergoCub forearm, to be used for shoulders FT characterization. Will start working on the design once I will have all the necessary infos.
• Will work for the (possible) VLMaps demo on Festival della Scienza on R1: few and minor fixes are needed, mostly visual stuff.
• After ICRA deadline, will update ergoCub to new YARP 3.10 and start working on the speech pipeline and porting R1 tour-guide.
• Currently working on 3D lidar with turtlebot4 for 3D mapping.

October

• Finished deploying VLMaps with a real time visualizer (ROS2 pointcloud aligned with 2D nav map).
• Prepared software for ECCV and Festival della scienza demo for VLMaps.
• Acquired dataset with 3D lidar with turtlebot and another one while walking.

TODO

• Implement speech pipeline on ergoCub (YARP 3.10)
• Work on VLMap querying
• Restructure ergocub_navigation code, reorganize and reorder.