Digital Human Resources

The repo collects some useful resources for facial capture and digital human creation.

Academic Groups

1. disney
2. cgl.ethz Computer Graphics Laboratory of ETHZ
3. gvv.mpi of Christian Theobalt
4. tuebingen.mpg
5. Derek Bradley from Disney
6. Paul Debevec from USC and google

Papers

Survery

1. Oliver Klehm, Fabrice Rousselle, Marios Papas, Derek Bradley, Christophe Hery, Bernd Bickel, Wojciech Jarosz, Thabo Beeler. Recent advances in facial appearance capture[J]. Computer Graphics Forum, 2015, 34(2):709–733.

2. M. Zollhofer, J. Thies, P. Garrido, D. Bradley, T. Beeler, P. Pérez, M. Stamminger, M. Nießner, C. Theobalt. State of the art on monocular 3d face reconstruction, tracking, and applications[J]. Computer Graphics Forum, 2018, 37(2):523–550.

Paul Debevec et.al light stage

1. Paul E. Debevec, Tim Hawkins, Chris Tchou, Haarm Pieter Duiker, Westley Sarokin, Mark Sagar. Acquiring the reflectance field of a human face[C]. In Proceedings of the 27th annual conference on Computer Graphics and Interactive Techniques. 2000.

2. Wan Chun Ma, Tim Hawkins, Pieter Peers, Charles Felix Chabert, Paul E. Debevec. Rapid acquisition of specular and diffuse normal maps from polarized spherical gradient illumination[C]. In Eurographics Conference on Rendering Techniques. 2007.

3. Abhijeet Ghosh, Tongbo Chen, Pieter Peers, Cyrus A. Wilson, Paul Debevec. Estimating specular roughness and anisotropy from second order spherical gradient illumination[J]. Computer Graphics Forum, 2009, 28(4):1161–1170.

4. Abhijeet Ghosh, Graham Fyffe, Borom Tunwattanapong, Jay Busch, Xueming Yu, Paul Debevec. Multiview face capture using polarized spherical gradient illumination[J]. ACM Transactions on Graphics, 2011, 30(6):1–10.

5. Christos Kampouris, Stefanos Zafeiriou, Abhijeet Ghosh. Diffuse-specular separation using binary spherical gradient illumination[C]. In Eurographics Symposium on Rendering - Experimental Ideas and Implementations. 2018, 1–10

6. G. Fyffe, P. Graham, B. Tunwattanapong, A. Ghosh, P. Debevec. Near instant capture of high resolution facial geometry and reflectance[J]. Computer Graphics Forum, 2016, 35(2):353–363.

7. G. Fyffe, K. Nagano, L. Huynh, S. Saito, J. Busch, A. Jones, H. Li, P. Debevec. Multi-view stereo on consistent face topology[J]. Computer Graphics Forum, 2017, 36(2):295–309.

8. TC Sun, JT Barron, YT Tsai, ZX Xu, XM Yu, G. Fyffe, C. Rhemann, J. Busch, P. Debevec, R. Ramamoorth. Single image portrait relighting[J]. ACM Transactions on Graphics, 2019, 38(4):1–12.

9. A. Meka, C. Hane, R. Pandey, M. Zollhofer, S. Fanello, G. Fyffe, A. Kowdle, XM Yu, J. Busch, J. Dour-Garian, P. Denny, S. Bouaziz, P. Lincoln, M. Whalen, G. Harvey, J. Taylor, S. Izadi, A. Tagliasacchi, P. Debevec, C. Theobalt, J. Valentin, C. Rhemann. Deep reflectance fields highquality facial reflectance field inference from color gradient illumination[J]. ACM Transactions on Graphics, 2019, 38(4):1–12.

Thabo Beeler et.al

1. Thabo Beeler, Bernd Bickel, Paul Beardsley, Bob Sumner, Markus Gross. High-quality single-shot capture of facial geometry[J]. ACM Transactions on Graphics, 2010, 29(4):1–9.

2. Thabo Beeler, Fabian Hahn, Derek Bradley, Bernd Bickel, Paul Beardsley, Craig Gotsman, Robert W. Sumner, Markus Gross. High-quality passive facial performance capture using anchor frames[J]. ACM Transactions on Graphics, 2011, 30(4).

3. Chen Cao, Derek Bradley, Kun Zhou, Thabo Beeler. Real-time high-fidelity facial performance capture[J]. ACM Transactions on Graphics, 2015, 34(4):46:1–46:9.

4. Paulo Gotardo, Jérémy Riviere, Derek Bradley, Abhijeet Ghosh, Thabo Beeler. Practical dynamic facial appearance modeling and acquisition[J]. ACM Transactions on Graphics, 2018, 37(6):232:1–232:13.

5. Jérémy Riviere, Paulo Gotardo, Derek Bradley, Abhijeet Ghosh, Thabo Beeler. Single-shot highquality facial geometry and skin appearance capture[J]. ACM Transactions on Graphics, 2020, 39(4).

Companies

1. pinscreen enables the rapid creation of 3D digital avatars of oneself.
2. soulmachines focuses on AI-driven realistic digital human avatars.
3. neon of Samsung aims at artificial humans for interactive applications.
4. reblika is a character outsourcing studio that provides 3D character services for video games and feature films.
5. digitaldomain
6. 3lateral develops innovative technologies that enable digitization of human appearance and motion, and it is a part of Epic Games Inc.
7. 3dmd

Softwares

1. 3D Reconstruction

1. Photoscan
2. CaptureReality
3. R3DS Warp3D and Warp4D
4. ZBrush
5. Xgen for hair modeling

2. Performance Capture

1. Disney Facial Performance Capture System named Medusa consists of a mobile rig of cameras and lights coupled with proprietary software that can reconstruct actor’s faces in full motion, without using traditional motion-capture dots.
2. Human and motion capture Noitom

3. Rendering

1. Arnold
2. Keyshot real-time 3D rendering

Blogs

1. Markus Gross The Future Of AI - Digital Humans Enter Their Primetime:
2. Jingyi Zhang
3. SoulShell from BIT
4. Doug Roble TED DigiDoug from digitaldomain
5. A STEP-BY-STEP BREAKDOWN OF 4D SCANNING from reblika and pixellighteffect

Codes

1. Implicit Skinning
2. PMVS the patch-based MVS algorithm to generate dense point cloud
3. MVS-Texturing Algorithm to texture 3D reconstructions from multi-view stereo images