An embodiment of a semiconductor package apparatus may include technology to perform depth sensor fusion to determine depth information for a surface, smooth the depth information for the surface and preserve edge information for the surface based on adaptive smoothing with self-tuning band-width estimation, iteratively remove holes from the surface based on conditional iterative manifold interpolation, reduce one or more of a file size and an on-memory storage size of data corresponding to the surface based on triangular edge contraction, and construct at least a portion of a 3D model based on data corresponding to a visible portion of the surface. Other embodiments are disclosed and claimed.