Skip to content

Commit

Permalink
Merge pull request RoBorregos#33 from RoBorregos/vision
Browse files Browse the repository at this point in the history
Added vision spotlights
  • Loading branch information
Ale-Coeto authored Dec 21, 2024
2 parents 98ba37f + b9ae773 commit 4e233e9
Show file tree
Hide file tree
Showing 2 changed files with 131 additions and 0 deletions.
120 changes: 120 additions & 0 deletions docs/development/vision/overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
# Node Overview

## General Nodes

### Object Detector (Trained Model) - Benvenuto
- Objects specified in the rulebook
- Objects given in competition

### Object Detector (Meta) - Joce
- Potentially new objects (other tools)

### Object Detector (3D)

### Face Recognition - Ale
- Recognize known faces
- Save new face

### Person Tracker - Ale
- Single tracker (track one person)
- By toggle
- By pose/gesture/pose
- Multi tracker (keep track of multiple people)

---

## Task-Specific Nodes

### GPSR
- gpsr_commands
- Person counting
- Object counting
- *Face recognition*
- *Person tracking*

### Carry My Luggage
- carry_commands
- Pointing detection
- Bag detection
- *Person tracking*

### Receptionist
- receptionist_commands
- Seat detection
- Person detection
- *Face recognition*

### Breakfast
- Object detector 2D
- Object detector 3D

### Storing Groceries
- groceries_commands
- List objects (moondream)
- *Object detector 2D*
- *Object detector 3D*

### Stickler for the Rules
- stickler_commands
- Trash detection
- Shoes on/off detection
- Cups/drinks detection
- Person detection
- *Person tracking*

### Restaurant
- restaurant_commands
- Customer detection (waving)
- *Object detection*
- *Person tracking*

---

## Utils (Classes)

### Pose/Gesture/Clothes Detector - Gil
- Given an image with a person, return:
- Pose
- Gesture
- Clothes and color
- Given an image with a person, return:
- Is visible (is chest visible)
- Center of the person (chest)
- Angle of the person

### Moondream2 Wrapper - Joce
- Given a prompt and an image, return the output
- Check fastest way to import moondream (ollama, transformers, etc)

---

## Details

### Gestures
- "waving person"
- "person raising their left arm"
- "person raising their right arm"
- "person pointing to the left"
- "person pointing to the right"

### Poses
- "sitting person"
- "standing person"
- "lying person"

### Types of Clothes
- "t-shirt"
- "shirt"
- "blouse"
- "sweater"
- "coat"
- "jacket"

### Colors
- "blue"
- "yellow"
- "black"
- "white"
- "red"
- "orange"
- "gray"
11 changes: 11 additions & 0 deletions docs/development/vision/spotlights.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Weekly Spotlights

This page is a collection of weekly spotlights that highlight the progress of the vision team. Each spotlight is a summary of the work done by the team in a week.

## 2024-12-20
This weeks progress was focused on:
- Creating the general vision package structure.
- Starting development of:
- Pose/gesture/clothes detector.
- Moondream2 wrapper.
- Face recognition migration to ros2.

0 comments on commit 4e233e9

Please sign in to comment.