公开(公告)号：US20240161388A1

公开(公告)日：2024-05-16

申请号：US18281966

申请日：2021-04-13

Applicant: SHANGHAITECH UNIVERSITY

Inventor： Haimin LUO ,  Minye WU ,  Lan XU ,  Jingyi YU

IPC: G06T15/20 ,  G06T7/593 ,  G06T7/62 ,  G06T17/00

CPC classification number: G06T15/20 ,  G06T7/596 ,  G06T7/62 ,  G06T17/00 ,  G06T2200/04 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: A deep neural network based hair rendering system is presented to model high frequency component of furry objects. Compared with existing approaches, the present method can generate photo-realistic rendering results. An acceleration method is applied in our framework, which can speed up training and rendering processes. In addition, a patch-based training scheme is introduced, which significantly increases the quality of outputs and preserves high frequency details.

公开(公告)号：US20230273318A1

公开(公告)日：2023-08-31

申请号：US17884273

申请日：2022-08-09

Applicant: XIAMEN UNIVERSITY ,  SHANGHAITECH UNIVERSITY

Inventor： Cheng WANG ,  Jialian LI ,  Lan XU ,  Chenglu WEN ,  Jingyi YU

IPC: G01S17/89 ,  G01S17/86 ,  G06N20/00

CPC classification number: G01S17/89 ,  G01S17/86 ,  G06N20/00

Abstract: Described herein are systems and methods for training machine learning models to generate three-dimensional (3D) motions based on light detection and ranging (LiDAR) point clouds. In various embodiments, a computing system can encode a machine learning model representing an object in a scene. The computing system can train the machine learning model using a dataset comprising synchronous LiDAR point clouds captured by monocular LiDAR sensors and ground-truth three-dimensional motions obtained from IMU devices. The machine learning model can be configured to generate a three-dimensional motion of the object based on an input of a plurality of point cloud frames captured by a monocular LiDAR sensor.

公开(公告)号：US20240273672A1

公开(公告)日：2024-08-15

申请号：US18568316

申请日：2021-07-12

Applicant: SHANGHAITECH UNIVERSITY

Inventor： Jingyi YU ,  Yuyao ZHANG ,  Lan XU ,  Yuwei LI ,  Qing WU

IPC: G06T3/4046 ,  G06T3/4053 ,  G06T5/10 ,  G06T5/50 ,  G06T5/60

CPC classification number: G06T3/4046 ,  G06T3/4053 ,  G06T5/10 ,  G06T5/50 ,  G06T5/60 ,  G06T2207/10061 ,  G06T2207/10081 ,  G06T2207/10088 ,  G06T2207/20048 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/30016

Abstract: Described herein are methods and non-transitory computer-readable media configured to obtain a plurality of images from a plurality of image scanning orientations for an object. A rigid registration is performed to the plurality of images to obtain a transformation matrix to normalize the plurality of images from their respective image spaces to a normalized image space. Each normalized image comprises a plurality of voxels. A machine learning model comprising an implicit representation of a high-resolution image is trained using the normalized images, wherein the high-resolution image comprises more voxels than the voxels in the normalized images. The high-resolution image is generated based on the trained machine learning model. The plurality of images are a plurality of anisotropic 2D images, while the high resolution image can be a 2D or 3D high resolution image.

公开(公告)号：US20240290059A1

公开(公告)日：2024-08-29

申请号：US18571748

申请日：2021-07-26

Applicant: SHANGHAITECH UNIVERSITY

Inventor： Jiakai ZHANG ,  Jingyi YU ,  Lan XU

IPC: G06V10/25 ,  G06T3/02 ,  G06T7/50

CPC classification number: G06V10/25 ,  G06T3/02 ,  G06T7/50

Abstract: A computer-implemented method of generating editable free-viewport videos is provided. A plurality of video of a scene from a plurality of views is obtained. The scene comprises includes an environment and one or more dynamic entities. A 3D bounding-box is generated for each dynamic entity in the scene. A computer device encodes a machine learning model including an environment layer and a dynamic entity layer for each dynamic entity in the scene. The environment layer represents a continuous function of space and time of the environment. The dynamic entity layer represents a continuous function of space and time of the dynamic entity. The dynamic entity layer includes a deformation module and a neural radiance module. The deformation module is configured to deform a spatial coordinate in accordance with a timestamp and a trained deformation weight. The neural radiance module is configured to derive a density value and a color.

公开(公告)号：US20230360372A1

公开(公告)日：2023-11-09

申请号：US18223575

申请日：2023-07-19

Applicant: SHANGHAITECH UNIVERSITY

Inventor： Fuqiang ZHAO ,  Minye WU ,  Lan XU ,  Jingyi YU

IPC: G06V10/774 ,  G06V10/82 ,  G06T7/50 ,  G06T7/80 ,  G06V10/74 ,  G06V20/64

CPC classification number: G06V10/774 ,  G06V10/82 ,  G06T7/50 ,  G06T7/80 ,  G06V10/761 ,  G06V20/64 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/10028 ,  G06V40/172

Abstract: Systems, methods, and non-transitory computer-readable media are configured to obtain a set of content items to train a neural radiance field-based (NeRF-based) machine learning model for object recognition. Depth maps of objects depicted in the set of content items can be determined. A first set of training data comprising reconstructed content items depicting only the objects can be generated based on the depth maps. A second set of training data comprising one or more optimal training paths associated with the set of content items can be generated based on the depth maps. The one or more optimal training paths are generated based at least in part on a dissimilarity matrix associated with the set of content items. The NeRF-based machine learning model can be trained based on the first set of training data and the second set of training data.