Add ball shooting into the Docking mission

[cbrxyz]

What needs to change?

The 2024 Docking mission will need to shoot balls into the holes in the dock. This includes:

• Recognizing the positions of the holes
• Aligning the boat to be able to consistently shoot the balls into the holes
• Calling the ball shooter driver to shoot the balls

How would this task be tested?

1. Ensure that balls are being shot into the docking holes in simulation.
2. Ensure that balls are being shot into the docking holes in real life.

[danushsingla]

This is our plan for this issue which is subject to change. Anthony is working with Alex's code currently on detecting the squares and integrating that into the program.

[Josht8601]

This week I met with Danush and we finalized everything for the cv model for detecting the color of the docking bays. We started working on the ball shooting for docking mission. This week we created a flow chart discussing what needs to be done for this task. As of know, I plan to work with the transform (tf) to map the ball shooter to the camera. Also this last week and this upcoming week are really hectic weeks due to midterm exams and projects due.

[AlexanderWangY]

I am starting to work on the ball shooter as well. Although I was not able to meet in the lab this week due to exams and school work, I have a general idea of what is happening and I will help with detecting the squares and returning the coordinates. I already have written some square detection using openCV which may be used to find the smaller squares as well.

[danushsingla]

I added Alex's code to the docking.py file and modified it so that it specifically looked for three squares and outlined them in the image. Some modifications will still need to be made for this, such as ensuring that those three squares are the squares we need to look for. Another problem is that it uses the old cropped image from when the submarine is looking to dock, when instead I think getting a new image once it's docked would be better.

[AlexanderWangY]

Due to the hurricane I was not able to finish much work this week. However, the new plan that Danush has to get a new image once docked would be good as then we can get a better and hopefully more accurate and defined picture of the dock. The square detection function should work better if we are able to isolate the board and then map out the points.

Also, I have begun making and testing generated test images with 3 squares instead of the singular colored square from the original docking mission. We should be able to extract and then pull the largest and smallest squares based on side length.

[Josht8601]

Danush and I met in the lab on tuesday to try and test Alex's code. We were trying to run and test it on the simulation on Will's laptop. We were testing using the 3 edges to try and see of it could detect the squares for the ball launcher. However due to the hurricane, I wasn't able to get much work done for the rest of the week. This week I plan to look more into the machine learning code to resolve some of the current issues related to the image cropping.

[danushsingla]

I was unable to come to the lab or do much work this week on the docking mission due to exams. I will have more time this upcoming week so I should be in the lab for more days and I intend on making significant progress.

[AlexanderWangY]

I was unable to due too much this week due to exams. After wed this upcoming week I should also have more time. In the mean time I have been staying up to date with the mission's progress and our next steps forward.

[Josht8601]

I was unable to come to the lab or do much work this week due to midterm exams and major coding projects that were due. I will be able to contribute more this week on Wednesday onwards all my major projects and exams will be completed by then. I intend to make more progress towards the mission this week.

[Josht8601]

This week I looked at the CV code for the docking bays and helped modify it to allow the boat to align better with each docking bay for when it come to the ball launcher and shooting the ball through one of the targets. Based on the current code, the boat wasn't able to fully align itself with the docking bays as it would hover back and forth near them. I also worked with Danush and Will to evaluate the motion of the boat around the docking bays in the simulation.

[danushsingla]

Due to the dock moving in the new world, Will, Josh, and I had to work on aligning the submarine properly. I personally worked on fixing the position of the submarine when docking, such as pushing it a little further back so that it can see the holes properly. Furthermore, I captured a second, clearer image that is able to take into account the holes. From here, the OpenCV code should work in detecting the three squares (two holes and colored square). This is the last thing to test and optimize. Fingers crossed, we should get this done by this week.

[AlexanderWangY]

Not much was done this week on my end due to an influx of exams this week. I will be back in action the coming week and will be helping out on preparing everything for the finished mission task.

[danushsingla]

Will and I managed to fix the simulation to the point where it was, for most of the time, docking correctly and taking a picture. However, we encountered some issues with the pictures and, again, the simulation was having the boat rock a lot. Unfortunately, this meant we were not able to fix the docking and shooting mission in time for the competition. We hope to continue working on this, nonetheless, and have the ball shooter be accurate. The next steps would be to understand why the images sometimes do not show up after taking one, and ensuring that the holes are detected appropriately as there were some issues with this.

[willzoo]

Update: As Danush said, we got the boat docking correctly the vast majority of the time. I've added comments to clarify what adjustments might need to be made to get it working at the practice course if it does not work initially. The ball shooting isn't working in the sim, but once you guys get to the practice course the boat should generally dock into the same position relative to the backboard, and assuming the ball always follows the same trajectory, a few hard-coded move commands after docking could align the racquet launcher with the hole.

[Josht8601]

This week I worked with Danish and Will to analyze some issues with the motion of the boat as it heads towards the docking bays when running the simulation. As Danush mentioned, the boat was rocking a lot in the simulation. We worked to try to figure out why the boat was wobbling a lot when running the simulation. Towards the end of the week I wasn’t able to come into the lab as I had some interviews scheduled.

[danushsingla]

I did not do anything for this issue this week because of the competition and also I was busy with other things. I will look to continue working on this, if it is still needed, after the competition.

[Josht8601]

I also didn’t do anything for the issue this week as the competition was going on during the week. I was also busy with other assignments and job interviews. I plan to continue working on this task if needed since the competition has now passed.