Shape reconstruction from images is one of the most widely adopted approaches to compute accurate 3D reconstructions of people or objects in a multi-camera environment. However, such algorithms are traditionally very sensitive to errors in the silhouettes due to imperfect foreground-background estimation or occluding objects appearing between the camera and the object of interest. We propose a novel algorithm that is still able to provide high quality reconstruction from incomplete silhouettes. At the core of the method is the partitioning of the reconstruction space in cells, i.e. regions with uniform camera and silhouette coverage properties. An iterative process is proposed which incrementally adds cells to the temporal reconstruction based on their potential to explain the observed silhouettes from different cameras. Experimental results are close to manually labelled approaches and outperform standard leave-M-out reconstruction techniques in terms of F1-score.
SlembrouckMVan CauwelaertDVan HammeDVan HaerenborghDVan HesePVeelaertP, et al. Self-learning voxel-based multi-camera occlusion maps for 3D reconstruction. In: International Conference on Computer Vision Theory and Applications, Proceedings. SCITEPRESS, 2014, p. 8.
2.
SlembrouckMVan CauwelaertDVeelaertPPhilipsW. Shape-from-silhouettes algorithm with built-in occlusion detection and removal. In: International Conference on Computer Vision Theory and Applications, Proceedings. SCITEPRESS, 2015.
3.
SlembrouckMVeelaertPVan HammeDVan CauwelaertDPhilipsW. Cell-based approach for 3D reconstruction from incomplete silhouettes. In: P. and Veelaert. Springer. 2017, p. 12.
4.
KangSBSzeliskiRChaiJ. Handling occlusions in dense multi-view stereo. In: Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on. vol. 1. IEEE, 2001. pp. 1-103.
5.
WegnerKStankiewiczODomańskiM. Occlusion handling in depth estimation from multiview video. In: Signals and Electronic Systems (ICSES), 2014 International Conference on. IEEE, 2014, pp. 1-4.
6.
ZitnickCLKanadeT. A cooperative algorithm for stereo matching and occlusion detection. IEEE Transactions on Pattern Analysis and Machine Intelligence.2000; 22(7): 675-684.
7.
SunJLiYKangSBShumHY. Symmetric stereo matching for occlusion handling. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05). vol. 2. IEEE, 2005. pp. 399-406.
8.
DingPSongY. Robust object tracking using color and depth images with a depth based occlusion handling and recovery. In: Fuzzy Systems and Knowledge Discovery (FSKD), 2015 12th International Conference on. IEEE, 2015. pp. 930-935.
9.
WuSZhangGZhuMJiangTNeriF. Geometry based three-dimensional image processing method for electronic cluster eye. Integrated Computer-Aided Engineering.2018; (Preprint): 1-16.
10.
FavaroPDuciAMaYSoattoS. On exploiting occlusions in multiple-view geometry. In: Computer Vision, 2003. Proceedings. Ninth IEEE International Conference on. IEEE, 2003. pp. 479-486.
11.
XiangYSavareseS. Object Detection by 3D Aspectlets and Occlusion Reasoning. In: 2013 IEEE International Conference on Computer Vision Workshops, 2013. pp. 530-537.
12.
GirshickRBFelzenszwalbPFMcallesterDA. Object detection with grammar models. In: Advances in Neural Information Processing Systems, 2011, pp. 442-450.
13.
MathiasMBenensonRTimofteRVan GoolL. Handling occlusions with franken-classifiers. In: Proceedings of the IEEE International Conference on Computer Vision,2013, pp. 1505-1512.
14.
DuanGAiHLaoS. A structural filter approach to human detection. In: European Conference on Computer Vision. Springer, 2010, pp. 238-251.
15.
OuyangWWangX. A discriminative deep model for pedestrian detection with occlusion handling. In: Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on. IEEE, 2012, pp. 3258-3265.
16.
OuyangWZengXWangX. Partial occlusion handling in pedestrian detection with a deep model, 2015.
17.
PosseggerHMauthnerTRothPMBischofH. Occlusion Geodesics for Online Multi-Object Tracking. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014.
18.
TopçuOAlatanAAErcanAO. Occlusion-aware 3D multiple object tracker with two cameras for visual surveillance. In: Advanced Video and Signal Based Surveillance (AVSS), 2014 11th IEEE International Conference on, 2014. pp. 56-61.
19.
JungJYoonIPaikJ. Object Occlusion Detection Using Automatic Camera Calibration for a Wide-Area Video Surveillance System. Sensors.2016; 16(7): 982s. Available from: http//www.mdpi.com/1424-8220/16/7/982.
20.
OtsukaKMukawaN. Multiview occlusion analysis for tracking densely populated objects based on 2-D visual angles. In: Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on. vol. 1, 2004. p. I–90-I–97.
21.
EssASchindlerKLeibeBVan GoolL. Improved multi-person tracking with active occlusion handling. In: ICRA Workshop on People Detection and Tracking. vol. 2 Citesee, 2009.
22.
ToyouraMIiyamaMFunatomiTKakushoKMinohM. 3d shape reconstruction from incomplete silhouettes in multiple frames. In: Pattern Recognition, 2008. ICPR 2008. 19th International Conference on. IEEE, 2008. pp. 1-4.
23.
GuanLSinhaSFrancoJSPollefeysM. Visual hull construction in the presence of partial occlusion. In: 3D Data Processing, Visualization, and Transmission, Third International Symposium on. IEEE, 2006. pp. 413-420.
24.
LandabasoJLPardàsMCasasJR. Shape from inconsistent silhouette. Computer Vision and Image Understanding.2008; 112(2): 210-224.
25.
HaroGPardàsM. Shape from incomplete silhouettes based on the reprojection error. Image and Vision Computing.2010; 28(9): 1354-1368.
26.
Díaz-MásLMadrid-CuevasFJMuñoz-SalinasRCarmona-PoyatoAMedina-CarnicerR. An octree-based method for shape from inconsistent silhouettes. Pattern Recognition.2012; 45(9): 3245-3255.
27.
Díaz-MásLMuñoz-SalinasRMadrid-CuevasFJMedina-CarnicerR. Shape from silhouette using Dempster – Shafer theory. Pattern Recognition.2010; 43(6): 2119-2131.
28.
PoikolainenINeriFCaraffiniF. Cluster-Based Population Initialization for differential evolution frameworks. Information Sciences.2015; 297: 216-235. Available from: shttp//www.sciencedirect.com/science/article/pii/S0020025514010962.
29.
LaurentiniA. The visual hull concept for silhouette-based image understanding. Pattern Analysis and Machine Intelligence, IEEE Transactions on.1994; 16(2): 150-162.
30.
BradskiG. The OpenCV Library. Dr Dobb’s Journal of Software Tools, 2000.
31.
RusuRBCousinsS. 3D is here: Point Cloud Library (PCL). In: IEEE International Conference on Robotics and Automation (ICRA). Shanghai, China, 2011.
32.
StarckJHiltonA. Surface capture for performance-based animation. Computer Graphics and Applications, IEEE.2007; 27(3): 21-31.
