signlanguage-detector

How to run..

Installation

The repository contains several example pictures and a trained model ( on MRFSegmentation and HoG features). These are used for the web cam application and the examples. To retrain a model etc. the datasets linked further down need to be downloaded.

Required packages

• python3 with numpy, sklearn, PyMaxFlow
• opencv2
• The trained model file needs to be downloaded from here
• Ideally its stored at resource/models/model_hog_asl.pkl

Runnables

• app.main runs the web cam application. Optimal conditions are:

  • light source from behind the camera
  • no other moving object/ object with skin colour inside the green box
  • when ready to calibrate move your hand slowly in the box and press "c"

• evaluation.plots.cthresh_asl shows example for colour threshold segmentation on ASL-set

• evaluation.plots.cthresh_tm shows example for colour threshold segmentation on TM-set

• evaluation.plots.mrf_asl shows example for Markov Random Field segmentation on ASL-set

• evaluation.plots.hog_asl shows example for HoG-descriptor on ASL-set

Utilities

• store.codebook generates Bag-of-HoGs codebook
• store.descriptors saves generated descriptors
• store.models trains model from data and saves it
• store.skinhist_asl generates average histogram of asl set and saves it
• test. contains tests for various parts

Datasets

Fingerspelling-Dataset - ASL

Fingerspelling Dataset - Thomas Moeslund

Report