Skip to content

Commit

Permalink
Face recognition initial PR (#221)
Browse files Browse the repository at this point in the history
This is the new pkg of face recognition using openface neural network
for face detection and recognition.
This is an initial PR checking.
  • Loading branch information
MatthijsBurgh authored Mar 12, 2024
2 parents e807d54 + ab780ee commit 283c302
Show file tree
Hide file tree
Showing 12 changed files with 742 additions and 0 deletions.
4 changes: 4 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -38,3 +38,7 @@
path = image_recognition_pose_estimation/docs
url = https://github.com/tue-robotics/tue_documentation_python.git
branch = master
[submodule "image_recognition_face_recognition/docs"]
path = image_recognition_face_recognition/docs
url = https://github.com/tue-robotics/tue_documentation_python.git
branch = master
1 change: 1 addition & 0 deletions image_recognition/package.xml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
<exec_depend>image_recognition_age_gender</exec_depend>
<exec_depend>image_recognition_analysis</exec_depend>
<exec_depend>image_recognition_color_extractor</exec_depend>
<exec_depend>image_recognition_face_recognition</exec_depend>
<exec_depend>image_recognition_footwear</exec_depend>
<exec_depend>image_recognition_msgs</exec_depend>
<exec_depend>image_recognition_openface</exec_depend>
Expand Down
20 changes: 20 additions & 0 deletions image_recognition_face_recognition/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
cmake_minimum_required(VERSION 3.0.2)
project(image_recognition_face_recognition)


find_package(catkin REQUIRED COMPONENTS)

catkin_python_setup()

catkin_package()

install(PROGRAMS
scripts/get_face_recognition
scripts/face_recognition_node
scripts/train_from_images
DESTINATION ${CATKIN_PACKAGE_BIN_DESTINATION}
)
if(CATKIN_ENABLE_TESTING)
find_package(catkin_lint_cmake REQUIRED)
catkin_add_catkin_lint_test()
endif()
70 changes: 70 additions & 0 deletions image_recognition_face_recognition/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# image_recognition_face_recognition

TO DO

Face recognition with use of Facenet (https://github.com/timesler/facenet-pytorch/tree/master)
Paper can be found: [paper](https://arxiv.org/pdf/1503.03832.pdf)

## Installation

See https://github.com/tue-robotics/image_recognition

Make sure you have installed **CUDA8**.

bash commends will be available when merged

## How-to

### Run Face Recognition Node

Run the command why using your camera as an input (use roscamera node):

rosrun cv_camera cv_camera_node
rosrun image_recognition_face_recognition face_recognition_node image:=/cv_camera/image_raw

### ROS Node

Run the image_recognition_face_detection node in one terminal (Specify the dlib and openface_net path as ROS parameter):

rosrun image_recognition_face_detection face_recognition_node

Run the rqt annotation client (https://github.com/tue-robotics/image_recognition_rqt)

rosrun image_recognition_rqt annotation_gui

Setup the service by clicking the gear wheel in the top-right corner. Select the `/annotate` services exposed by the openface ros node. Also don't forget to set-up your labels.

![Configuration](doc/config.png)

Now draw a rectangle around the face you would like to learn. The face recognizer will find the biggest face in the image and store a representation for this face.

![Annotate](doc/annotate.png)

Now select the label and you will see that the openface ros node stores the face (console output node):

[INFO] [WallTime: 1478636380.407308] Succesfully learned face of 'rokus'

Learn as many faces as you want ..

Next step is starting the image_recognition_Rqt test gui (https://github.com/tue-robotics/image_recognition_rqt)

rosrun image_recognition_rqt test_gui

Again configure the service you want to call with the gear-wheel in the top-right corner of the screen. If everything is set-up, draw a rectangle in the image and ask the service for detections:

![Test](doc/test_face.png)

You will see that the result of the detection will prompt in a dialog combo box. Also the detections will be drawn on the image.


### Command line

Command line interface to test the detection / recognition based on an image:

usage: get_face_recognition IMAGE [-k ALIGN_PATH] [-s NET_PATH] [-v]

Run the command on an example image:

rosrun image_recognition_face_recognition get_face_recognition `rospack find image_recognition_face_recognition`/doc/example.png

![Example](doc/example.png)
1 change: 1 addition & 0 deletions image_recognition_face_recognition/docs
Submodule docs added at e81558
38 changes: 38 additions & 0 deletions image_recognition_face_recognition/package.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
<?xml version="1.0"?>
<?xml-model
href="http://download.ros.org/schema/package_format3.xsd"
schematypens="http://www.w3.org/2001/XMLSchema"?>
<package format="3">
<name>image_recognition_face_recognition</name>
<version>0.0.1</version>
<description>The image_recognition_face_recognition package</description>

<maintainer email="[email protected]">iason</maintainer>

<license>MIT</license>

<buildtool_depend>catkin</buildtool_depend>

<buildtool_depend>python3-setuptools</buildtool_depend>

<exec_depend>roscpp</exec_depend>
<exec_depend>cv_bridge</exec_depend>
<exec_depend>diagnostic_updater</exec_depend>
<exec_depend>image_recognition_msgs</exec_depend>
<exec_depend>image_recognition_util</exec_depend>
<exec_depend>python-dlib</exec_depend> <!--pip package-->
<exec_depend>python3-numpy</exec_depend>
<exec_depend>python3-opencv</exec_depend>
<exec_depend>rospy</exec_depend>

<test_depend>catkin_lint_cmake</test_depend>

<doc_depend>python3-sphinx</doc_depend>
<doc_depend>python-sphinx-autoapi-pip</doc_depend>
<doc_depend>python-sphinx-rtd-theme-pip</doc_depend>
<doc_depend>python3-yaml</doc_depend>

<export>
<rosdoc config="rosdoc.yaml" />
</export>
</package>
3 changes: 3 additions & 0 deletions image_recognition_face_recognition/rosdoc.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
- builder: sphinx
sphinx_root_dir: docs
name: Python API
Loading

0 comments on commit 283c302

Please sign in to comment.