forked from RoBorregos/Home-Docs
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request RoBorregos#33 from RoBorregos/vision
Added vision spotlights
- Loading branch information
Showing
2 changed files
with
131 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,120 @@ | ||
# Node Overview | ||
|
||
## General Nodes | ||
|
||
### Object Detector (Trained Model) - Benvenuto | ||
- Objects specified in the rulebook | ||
- Objects given in competition | ||
|
||
### Object Detector (Meta) - Joce | ||
- Potentially new objects (other tools) | ||
|
||
### Object Detector (3D) | ||
|
||
### Face Recognition - Ale | ||
- Recognize known faces | ||
- Save new face | ||
|
||
### Person Tracker - Ale | ||
- Single tracker (track one person) | ||
- By toggle | ||
- By pose/gesture/pose | ||
- Multi tracker (keep track of multiple people) | ||
|
||
--- | ||
|
||
## Task-Specific Nodes | ||
|
||
### GPSR | ||
- gpsr_commands | ||
- Person counting | ||
- Object counting | ||
- *Face recognition* | ||
- *Person tracking* | ||
|
||
### Carry My Luggage | ||
- carry_commands | ||
- Pointing detection | ||
- Bag detection | ||
- *Person tracking* | ||
|
||
### Receptionist | ||
- receptionist_commands | ||
- Seat detection | ||
- Person detection | ||
- *Face recognition* | ||
|
||
### Breakfast | ||
- Object detector 2D | ||
- Object detector 3D | ||
|
||
### Storing Groceries | ||
- groceries_commands | ||
- List objects (moondream) | ||
- *Object detector 2D* | ||
- *Object detector 3D* | ||
|
||
### Stickler for the Rules | ||
- stickler_commands | ||
- Trash detection | ||
- Shoes on/off detection | ||
- Cups/drinks detection | ||
- Person detection | ||
- *Person tracking* | ||
|
||
### Restaurant | ||
- restaurant_commands | ||
- Customer detection (waving) | ||
- *Object detection* | ||
- *Person tracking* | ||
|
||
--- | ||
|
||
## Utils (Classes) | ||
|
||
### Pose/Gesture/Clothes Detector - Gil | ||
- Given an image with a person, return: | ||
- Pose | ||
- Gesture | ||
- Clothes and color | ||
- Given an image with a person, return: | ||
- Is visible (is chest visible) | ||
- Center of the person (chest) | ||
- Angle of the person | ||
|
||
### Moondream2 Wrapper - Joce | ||
- Given a prompt and an image, return the output | ||
- Check fastest way to import moondream (ollama, transformers, etc) | ||
|
||
--- | ||
|
||
## Details | ||
|
||
### Gestures | ||
- "waving person" | ||
- "person raising their left arm" | ||
- "person raising their right arm" | ||
- "person pointing to the left" | ||
- "person pointing to the right" | ||
|
||
### Poses | ||
- "sitting person" | ||
- "standing person" | ||
- "lying person" | ||
|
||
### Types of Clothes | ||
- "t-shirt" | ||
- "shirt" | ||
- "blouse" | ||
- "sweater" | ||
- "coat" | ||
- "jacket" | ||
|
||
### Colors | ||
- "blue" | ||
- "yellow" | ||
- "black" | ||
- "white" | ||
- "red" | ||
- "orange" | ||
- "gray" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
# Weekly Spotlights | ||
|
||
This page is a collection of weekly spotlights that highlight the progress of the vision team. Each spotlight is a summary of the work done by the team in a week. | ||
|
||
## 2024-12-20 | ||
This weeks progress was focused on: | ||
- Creating the general vision package structure. | ||
- Starting development of: | ||
- Pose/gesture/clothes detector. | ||
- Moondream2 wrapper. | ||
- Face recognition migration to ros2. |