Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support of face alignment #110

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

jdongca2003
Copy link
Contributor

Face alignment plays an important role in facial analysis (e.g. emotion detection, face recognition). MTCNN has already extracted 5 facial landmark points. We can compare those landmark points with reference ones to construct an affine transform for face alignment. Here I use open source code from https://github.com/ZhaoJ9014/face.evoLVe.PyTorch/blob/master/align/align_trans.py

The changes are minimum:

  1. in MTCNN , add extract_aligned_face function
  2. add two files (align_trans.py and matlab_cp2tform.py under folder utils
  3. update dependency opencv-python in setup.py

With the above changes, I tested this feature. In my experiments, the aligned face size is 112 x 112. The accuracy gain is more than 1% in the internal celebrity dataset.

@AGenchev
Copy link

AGenchev commented Dec 29, 2020

@jdongca2003 I think the face alignment should be integrated into the MTCNN.forward() function and switched on with optional align_face argument or such argument in the MTCNN constructor.
I also don't see the "prewhiten" function used, though I would call it "normalize" or something like it. It might change the model output.

@jdongca2003
Copy link
Contributor Author

Hi Angel,
Sorry for delay. I agreed with your design (aligned as an argument in forward function). (align_trans.py) is from other repository though I documented it in header. This may not be good. You can add feature request for face align features. Hope that faceNet community developers can pick it up.

Thanks

Jianxiong

@arruw
Copy link

arruw commented Jan 26, 2021

I have share simple alignment script here: #34 (comment)
It is not optimal, but it is simple and readable 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants