This is an OpenCV project to
- Process the coloured drawing and extract the original line drawing
- Track a surface in a video
- Get the camera position
- Render a 3D object on to the video
The inspiration for this work was from a paper from Disney Research Magnenat et al (2015), 'Live Texturing of Augmented Reality Characters from Colored Drawings'. You can find more about the work here: https://www.disneyresearch.com/publication/live-texturing-of-augmented-reality-characters/
In this work I have used OpenCV library for implementing the project. Some of the key techniques I used are:
- Camera calibration (chessboard pattern)
- Extraction of line drawing from coloured drawings
- BRISK feature detector (cv2.BRISK_create, brisk.detectAndCompute)
- FLANN feature matching (cv2.FlannBasedMatcher, flann.knnMatch)
- Optical flow tracking (cv2.calcOpticalFlowPyrLK)
- Compute Homography (cv2.findHomography)
- Camera pose estimation (cv2.solvePnPRansac)
- Simple plotting of cube using cv2.line function
I have mainly relied on opencv documentations to understand how to use its functions and the above mentioned paper to implement this project. Special thanks to the wonderful resources on www.learnopencv.com run by @spmallick and also the blog article by @juangallostra (I see he has published Part 2 as well!)