3D human face transplanting via depth camera

[4] Jing Tong , Jin Zhou , Ligang Liu . Scanning 3D full human bodies using Kinects.. IEEE Trans Visual Comput Graphics, 2012, 18: 643-650 CrossRef PubMed Google Scholar

[5] Zhou J, Pan J J, Tong J, et al. Fast 3D human body reconstruction from Kinects. J Comput-Aid Des Comput Graph, 2013, 25: 873--879. Google Scholar

[6] Yang H Z, Lu Y, Fang Q, et al. Automatic 3D scanning system via depth camera. J Comput-Aid Des Comput Graph, 2015, 27: 2039--2045. Google Scholar

[7] Zhou W, Guan Z Q, Lu Y, et al. Easy 3D self-portraits. In: Proceedings of the 10th China Computer Graphics Conference and the 18th National CAGD Conference Papers. Hefei: USTC, 2014. 178. Google Scholar

[8] Chandawarkar A A, Diaz-Siso J R, Bueno E M. Facial appearance transfer and persistence after three-dimensional virtual face transplantation.. Plastic Reconstructive Surgery, 2013, 132: 957-966 CrossRef PubMed Google Scholar

[9] Pérez P, Gangnet M, Blake A. Poisson image editing. ACM Trans Graph, 2003, 22: 313-318 CrossRef Google Scholar

[10] Reinhard E, Ashikhmin M, Gooch B, et al. Color transfer between images. IEEE Comput Graph Appl, 2001, 21: 34--41. Google Scholar

[11] Lempitsky V, Ivanov D. Seamless mosaicing of image-based texture maps. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, 2007. Google Scholar

[12] Sorkine O, Cohen-Or D, Lipman Y, et al. Laplacian surface editing. In: Proceedings of the 2004 Eurographics/ACM SIGGRAPH Symposium on Geometry Processing, Nice, 2004. 175--184. Google Scholar

[13] Belkin M, Sun J, Wang Y. Discrete Laplace operator on meshed surfaces. In: Proceedings of the 24th Annual Symposium on Computational Geometry, College Park, 2008. Google Scholar

[14] Newcombe R A, Izadi S, Hilliges O, et al. KinectFusion: real-time dense surface mapping and tracking. In: Proceedings of the 10th IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Basel, 2011. 127--136. Google Scholar

[15] Kazhdan M, Hoppe H. Screened Poisson surface reconstruction. ACM Trans Graph, 2013, 32: 29. Google Scholar

[16] Kazhdan M, Bolitho M, Hoppe H. Poisson surface reconstruction. In: Proceedings of the 4th Eurographics Symposium on Geometry Processing, Cagliari, 2006. 61--70. Google Scholar

[17] Floater M S. Parametrization and smooth approximation of surface triangulations. Comput Aided Geometric Des, 1997, 14: 231-250 CrossRef Google Scholar

[18] Field D A. Laplacian smoothing and Delaunay triangulations. Int J Numer Method Biomed Eng, 1988, 4: 709--712. Google Scholar

Click to view Download high-quality image

Figure 1
(Color online) 3D face transplanting effects. (a) Testing subject; (b) transplanting based on an image; (c) result of this paper
Click to view Download high-quality image

Figure 2
(Color online) Symbolic description
Click to view Download high-quality image

Figure 3
(Color online) Face scanning
Click to view Download high-quality image

Figure 4
System flow
Click to view Download high-quality image

Figure 5
(Color online) Face extracting procedure. (a) Original point cloud; (b) crop via depth image; (c) crop via feature points; (d) cropping result; (e) geo-reconstruction; (f) texture reconstruction and feature points
Click to view Download high-quality image

Figure 6
(Color online) Geo-detail comparisons between Poisson and Screen Poisson reconstructions (${\rm~Depth}=8$). Details of (a) Poisson and (b) Screen Poisson reconstruction method
Click to view Download high-quality image

Figure 7
(Color online) Laplace vector
Click to view Download high-quality image

Figure 8
(Color online) Geo-warping of scanned face. (a) Rigid transform; (b) correspondence of boundary; (c) result of warping
Click to view Download high-quality image

Figure 9
(Color online) Rigid transform effects between 5 pairs and 78 pairs of feature points. (a) Orthographic view; (b) overhead view; (c) side view
Click to view Download high-quality image

Figure 10
(Color online) Boundary smoothing. (a) Before smoothing; (b) after smoothing
Click to view Download high-quality image

Figure 11
(Color online) Texturing effect comparisons. (a) Initial texture; (b) color optimization; (c) Poisson editing; (d) combine both methods
Click to view Download high-quality image

Figure 12
(Color online) Female character with male face
Click to view Download high-quality image

Figure 13
(Color online) Avatar with female face
Click to view Download high-quality image

Figure 14
(Color online) Cartoon model with human face
Click to view Download high-quality image

Figure 15
(Color online) Facial detail enhancement

Algorithm 1 自动三维面部提取算法

Require:人脸三维点云$C_s$, 采样$(I_c,I_d,P)$;
Output:裁剪后的人脸三角网格$F_s$.
借助采样数据进行点云的初步裁剪 (如图4(b)): 将深度图$I_d$转化为点云, 并借助相机位置$P$将点云注册到$C_s$位置处, 记为$C_d$, 定义$C_s$ 上一点$\boldsymbol{p}$到$C_d$的距离: $$d(\boldsymbol{p},C_d)={\rm min}_{\boldsymbol{q}\in{C_d}}{{\rm length}(\boldsymbol{p}-\boldsymbol{q})},$$ 给定阈值$d_{\rm~max}$, 若$d(\boldsymbol{p},C_d)>d_{\rm~max}$, 则此$\boldsymbol{p}$点舍弃.
借助特征点位置对点云进一步裁剪 (如图5(c)): 从采样图$I_c$可自动检测人脸特征点, 利用$I_c$ 与$I_d$的对应关系及相机位置$P$, 可获取$C_d$ 上特征点位置, 特征点的最低高度与最高高度分别设为$h_{\rm~min}$与$h_{\rm~max}$, 由于人脸基本处于竖直状态, 预设offset为0.01 m, 若$p_y<h_{\rm~min}-{\rm~offset}$或$p_y>h_{\rm~max}+{\rm~offset}$, 则此$\boldsymbol{p}$点舍弃.
使用Screen Poisson ^[15]表面重建算法提取三角网格 (如图5(e)).

Algorithm 2 面部网格变形算法

Require:$F_s$, $F_m$和$M_l$;
Output:与$M_l$贴合情况较好的人脸网格.
利用特征点刚性变换$F_s$至$F_m$ (图8(a));
寻找$F_s$与$M_l$人脸边缘的对应关系, 以确定人脸新的边界点位置 (图8(b));
使用基于网格Laplace的变形算法, 构造sect. 5.1小节中的线性系统求解顶点坐标, 变形$F_s$以适应$M_l$的面部边界轮廓 (图8(c)).

0

Citations
0

Altmetric
Article metrics

Email
Export

3D human face transplanting via depth camera

Abstract

Funded by

References

0

0

Interesting search

Top 5Related Articles