Multi-view 3D People Reconstruction combining Parametric and Non-parametric models

Author: Òscar Lorente Corominas

Virtual Room: https://eu.bbcollab.com/guest/12c492ad881f403dbcff1a9488b1da96

Date & time: 22/09/2021 – 10:15 h

Session name: Human Modelling

Supervisors: Xavier Giró-i-Nieto, Francesc Moreno-Noguer

Abstract:

3D reconstruction of human bodies from multiple images has been a long-standing problem in computer vision. It is typically addressed using statistical models of the human body, which describe the geometry by a small number of parameters encoding 3D pose and shape. Non-parametric representations are alternatives that gain expressiveness for cloth capture, but have difficulties in recovering reasonable 3D human shapes when camera views are too sparse. In this dissertation, we aim to leverage the advantages of parametric and non-parametric models by extending the parametric Skinned Multi-Person Linear Model (SMPL) with Implicit Differentiable Renderer (IDR), an architecture that implicitly represents the geometry as a zero level-set of a neural network. The neural surface of IDR is typically initialized as a sphere, which allows rendering objects of all types. However, our work focuses on the reconstruction of human bodies, so we explore the contribution of parametric 3D human models such as SMPL as priors. The evaluation has been performed on a subset of the Renderpeople dataset, using as metrics for 3D reconstruction the Chamfer-L1 and point-to-surface distances, as well as PSNR for the corresponding renderings. The obtained results confirm that in scenarios where the camera views are too sparse, using an SMPL model as a prior improves 3D reconstruction and accelerates convergence. Finally, we propose a strategy based on an attention mechanism for IDR to improve the results on the head of the person, where the original IDR pipeline struggles to achieve a detailed reconstruction.

Committee:

– President: Josep R. Casas(UPC)
– Secretary: Gloria Haro(UPF)
– Vocal: Federico Sukno(UPF)