公开(公告)号：US20250111474A1

公开(公告)日：2025-04-03

申请号：US18830914

申请日：2024-09-11

Applicant: NVIDIA Corporation

Inventor： Koki Nagano ,  Alexander Trevithick ,  Matthew Aaron Wong Chan ,  Towaki Takikawa ,  Umar Iqbal ,  Shalini De Mello

IPC: G06T3/4046 ,  G06T5/60 ,  G06T5/70 ,  G06T15/08

Abstract: Systems and methods are disclosed that relate to synthesizing high-resolution 3D geometry and strictly view-consistent images that maintain image quality without relying on post-processing super resolution. For instance, embodiments of the present disclosure describe techniques, systems, and/or methods to scale neural volume rendering to the much higher resolution of native 2D images, thereby resolving fine-grained 3D geometry with unprecedented detail. Embodiments of the present disclosure employ learning-based samplers for accelerating neural rendering for 3D GAN training using up to five times fewer depth samples, which enables embodiments of the present disclosure to explicitly “render every pixel” of the full-resolution image during training and inference without post-processing super-resolution in 2D. Together with learning high-quality surface geometry, embodiments of the present disclosure synthesize high-resolution 3D geometry and strictly view—consistent images while maintaining image quality on par with baselines relying on post-processing super resolution.

公开(公告)号：US20240104842A1

公开(公告)日：2024-03-28

申请号：US18472653

申请日：2023-09-22

Applicant: NVIDIA Corporation

Inventor： Koki Nagano ,  Alexander Trevithick ,  Chao Liu ,  Eric Ryan Chan ,  Sameh Khamis ,  Michael Stengel ,  Zhiding Yu

IPC: G06T17/00 ,  G06T5/20 ,  G06T7/70 ,  G06T7/90 ,  G06V10/771

CPC classification number: G06T17/00 ,  G06T5/20 ,  G06T7/70 ,  G06T7/90 ,  G06V10/771 ,  G06T2207/10024

Abstract: A method for generating, by an encoder-based model, a three-dimensional (3D) representation of a two-dimensional (2D) image is provided. The encoder-based model is trained to infer the 3D representation using a synthetic training data set generated by a pre-trained model. The pre-trained model is a 3D generative model that produces a 3D representation and a corresponding 2D rendering, which can be used to train a separate encoder-based model for downstream tasks like estimating a triplane representation, neural radiance field, mesh, depth map, 3D key points, or the like, given a single input image, using the pseudo ground truth 3D synthetic training data set. In a particular embodiment, the encoder-based model is trained to predict a triplane representation of the input image, which can then be rendered by a volume renderer according to pose information to generate an output image of the 3D scene from the corresponding viewpoint.