Structure-consistent customized virtual mannequin reconstruction from 3D scans based on optimization

Abstract

The 3D virtual mannequin has been widely used in apparel industry, and its importance is also increasing. This work develops a new 3D virtual mannequin reconstruction system based on optimization. All the mannequins reconstructed by the proposed approach share the identical topology, that is, there is a point-to-point correspondence among the mannequins, which will significantly facilitate much subsequent processing in fashion design, made-to-measure, and virtual try-on. The inputs to the proposed system contain a template human body, a raw scan (represented in mesh), and a very sparse corresponding landmarks set. The proposed approach substantially utilizes the optimization technology to drive the template to deform into a real scan. There is no special requirement on the raw meshes. The raw meshes may have a different number of vertices and triangles or may even be incomplete. The proposed method only needs 21 landmarks as hard-constraints to reconstruct a mannequin with tens of thousands of vertices. These landmarks can be extracted automatically for standard mannequin reconstruction. Besides the standard mannequin, the proposed system can also be used to reconstruct display mannequins, that is, mannequins with various poses. The experiments visualize the optimization procedure and verify that the optimization is efficient and effective. Quantitative analysis also proves that the reconstruction error satisfies the requirements of fashion design and tailoring.

Keywords

virtual mannequin mannequin reconstruction standard mannequin display mannequin mesh deformation

Get full access to this article

View all access options for this article.

References

Hsiao

S-W

Chen

R-Q

. A study of surface reconstruction for 3D mannequins based on feature curves. Comput-Aided Des 2013; 45: 1426–1441.

, et al. Personalized 3D mannequin reconstruction based on 3D scanning. Int J Cloth Sci Technol 2018; 30: 159–174.

Zhong

, et al. Automatic body measurement based on slicing loops. Int J Cloth Sci Technol 2018; 30: 380–397.

Wang

CCL

. Parameterization and parametric design of mannequins. Comput-Aided Des 2005; 37: 83–98.

Wang

CCL

Chang

TKK

Yuen

MMF

. From laser-scanned data to feature human model: a system based on fuzzy logic concept. Comput-Aided Des 2003; 35: 241–253.

Wei W, Luo X and Li Z. Layer-based mannequin reconstruction and parameterization from 3D range data. In: Advances in geometric modeling and processing. Berlin, Heidelberg: Springer, 2008, pp.498–504.

Hsiao

S-W

Chen

R-Q

. Applying Kinect on the development of a customized 3D mannequin. Evaluation 2015; 39: 61530–61530.

Huang

, et al. Body scanning and modeling for custom fit garments. J Text Appar Technol Manag 2002; 2: 1–11.

Zhong

. Automatic segmenting and measurement on scanned human body. Int J Cloth Sci Technol 2006; 18: 19–30.

10.

Loper

Mahmood

Romero

, et al. SMPL: a skinned multi-person linear model. ACM Trans Graph (TOG) 2015; 34: 248–248.

11.

Anguelov

Srinivasan

Koller

, et al. SCAPE: shape completion and animation of people. Int Conf Comput Graph Interact Tech 2005; 24: 408–416.

12.

Kanazawa A, Black MJ, Jacobs DW, et al. End-to-end recovery of human shape and pose. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp.7122–7131.

13.

Dibra E, Jain H, Öztireli C, et al. Hs-nets: estimating human body shape from silhouettes with convolutional neural networks. In: 2016 fourth international conference on 3D vision (3DV). IEEE, 2016, pp.108–117.

14.

Pishchulin L, Insafutdinov E, Tang S, et al. DeepCut: joint subset partition and labeling for multi person pose estimation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp.4929–4937.

15.

Dibra E, Jain H, Oztireli C, et al. Human shape from silhouettes using generative HKS descriptors and cross-modal neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp.4826–4836.

16.

Omran M, Lassner C, Pons-Moll G, et al. Neural body fitting: unifying deep learning and model-based human pose and shape estimation. In: 2018 international conference on 3D vision (3DV). 2018, pp.484–494.

17.

Ji Z, Qi X, Wang Y, et al. Shape-from-mask: a deep learning based human body shape reconstruction from binary mask images. ArXiv180608485 Cs, http://arxiv.org/abs/1806.08485 (2018, accessed 3 March 2019).

18.

Dibra E, Öztireli C, Ziegler R, et al. Shape from selfies: human body shape estimation using CCA regression forests. In: European conference on computer vision. Springer, 2016, pp.88–104.

19.

Agarwal A and Triggs B. Learning to track 3D human motion from silhouettes. In: Proceedings of the twenty-first international conference on machine learning. ACM, 2004, p.2.

20.

Alldieck T, Kassubeck M, Wandt B, et al. Optical flow-based 3D human motion estimation from monocular video. In: Roth V and Vetter T (eds) Pattern recognition. Springer International Publishing, 2017, pp.347–360.

21.

Huang Y, Bogo F, Lassner C, et al. Towards accurate marker-less human shape and pose estimation over time. In: 2017 international conference on 3D vision (3DV). 2017, pp.421–430.

22.

Rhodin H, Robertini N, Casas D, et al. General automatic human shape and motion capture using volumetric contour cues. In: European conference on computer vision. Springer, 2016, pp.509–526.

23.

Pishchulin

Wuhrer

Helten

, et al. Building statistical shape spaces for 3D human modeling. Pattern Recognit 2017; 67: 276–286.

24.

Chatterjee

Zollhoefer

, et al. MonoPerfCap: human performance capture from monocular video. ACM Trans Graph TOG 2018; 37: 27–27.

25.

Sigal L, Balan A and Black MJ. Combined discriminative and generative articulated pose and non-rigid shape estimation. In: Advances in neural information processing systems. Springer, 2008, pp.1337–1344.

26.

Guan P, Weiss A, Bãlan AO, et al. Estimating human shape and pose from a single image. In: IEEE international conference on computer vision, ICCV 2009, Kyoto, Japan, 27 September–4 October 2009, pp.1381–1388.

27.

Bogo F, Kanazawa A, Lassner C, et al. Keep it SMPL: automatic estimation of 3D human pose and shape from a single image. In: European conference on computer vision. Springer, 2016, pp.561–578.

28.

Lassner C, Romero J, Kiefel M, et al. Unite the people: closing the loop between 3D and 2D human representations. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 6050–6059.

29.

Kim

Park

. Parametric body model generation for garment drape simulation. Fiber Polym 2004; 5: 12–18.

30.

Huang

Kwok

T-H

Zhou

. Parametric design for human body modeling by wireframe-assisted deep learning. Comput-Aided Des 2019; 108: 19–29.

31.

Allen

Curless

Popović

. The space of human body shapes: reconstruction and parameterization from range scans. ACM Trans Graph 2003; 22: 587–594.

32.

Yang Y, Yu Y, Zhou Y, et al. Semantic parametric reshaping of human body models. In: 3D Vision (3DV), 2014 2nd International Conference on. IEEE, 2014, pp.41–48.

33.

Jakob

Tarini

Panozzo

, et al. Instant field-aligned meshes. ACM Trans Graph 2015; 34: 1–15.

34.

Zhong

, et al. Two-step registration in 3D human body scanning based on multiple RGB-D sensors. J Fiber Bioeng Inform 2015; 8: 705–712.

35.

Bronstein

Kimmel

. Efficient computation of isometry‐invariant distances between surfaces. SIAM J Sci Comput 2006; 28: 1812–1836.

36.

Chen

Golovinskiy

Funkhouser

. A benchmark for 3D mesh segmentation. ACM Trans Graph 2009; 28: 341–352.