gestures_recog_NN

hhh

PyTorch implementation of a CNN based gesture recognition system.

This repo is about the project I've done and presented for my Master's degree thesis. All the details and techincal analysis are explained in my thesis and presentation.

The whole thesis project leaded to the submission of the paper "Hand Gestures for the Human-Car Interaction:the Briareo dataset" Fabio Manganaro, Stefano Pini, Guido Borghi, Roberto Vezzani, and Rita Cucchiara. University of Modena and Reggio Emilia, Italy, Department of Engineering “Enzo Ferrari", submitted to ICIAP 2019.

Important

For more info about the project and instructions to download the dataset please refer to http://imagelab.ing.unimore.it/briareo.

Video example of working prototype:

Implemented Models

The gesture recogntion system exploits 4 different neural network architectures implemented in models.py and cross_mode_net.py

1. CNN 2D

2. Multi modal net

3. C3D

4. LSTM

Training Set

All the models have been trainend usinge the custom dataset called Briareo (read Hand Gestures for the Human-Car Interaction:the Briareo dataset) which has been collected using the following implemented framework: gestures collector.

Example of gestures in the dataset.

System prototype

The system prototype has been implementented in the project: demo_gesture_recog.

Contents

1. main.py provides all the CNN models which have been used and training sturucture.
2. dataloeder.py provides the implementation of pytorch dataloaed for a custom dataset.