STEP DISTILLATION FOR LATENT DIFFUSION MODELS

    公开(公告)号:US20240394843A1

    公开(公告)日:2024-11-28

    申请号:US18434411

    申请日:2024-02-06

    Applicant: Snap Inc.

    Abstract: Described is a system for improving machine learning models by accessing a first latent diffusion machine learning model, the first latent diffusion machine learning model trained to perform a first number of denoising steps, accessing a second latent diffusion machine learning model that was derived from the first latent diffusion machine learning model, the second latent diffusion machine learning model trained to perform a second number of denoising steps, generating noise data, processing the noise data via the first latent diffusion machine learning model to generate one or more first images, processing the noise data via the second latent diffusion machine learning model to generate one or more second images, and modify a parameter of the second latent diffusion machine learning model based on a comparison of the one or more first images with the one or more second images.

    Compressing image-to-image models with average smoothing

    公开(公告)号:US12154303B2

    公开(公告)日:2024-11-26

    申请号:US18238979

    申请日:2023-08-28

    Applicant: Snap Inc.

    Abstract: System and methods for compressing image-to-image models. Generative Adversarial Networks (GANs) have achieved success in generating high-fidelity images. An image compression system and method adds a novel variant to class-dependent parameters (CLADE), referred to as CLADE-Avg, which recovers the image quality without introducing extra computational cost. An extra layer of average smoothing is performed between the parameter and normalization layers. Compared to CLADE, this image compression system and method smooths abrupt boundaries, and introduces more possible values for the scaling and shift. In addition, the kernel size for the average smoothing can be selected as a hyperparameter, such as a 3×3 kernel size. This method does not introduce extra multiplications but only addition, and thus does not introduce much computational overhead, as the division can be absorbed into the parameters after training.

    MOTION REPRESENTATIONS FOR ARTICULATED ANIMATION

    公开(公告)号:US20210407163A1

    公开(公告)日:2021-12-30

    申请号:US17364218

    申请日:2021-06-30

    Applicant: Snap Inc.

    Abstract: Systems and methods herein describe novel motion representations for animating articulated objects consisting of distinct parts. The described systems and method access source image data, identify driving image data to modify image feature data in the source image sequence data, generate, using an image transformation neural network, modified source image data comprising a plurality of modified source images depicting modified versions of the image feature data, the image transformation neural network being trained to identify, for each image in the source image data, a driving image from the driving image data, the identified driving image being implemented by the image transformation neural network to modify a corresponding source image in the source image data using motion estimation differences between the identified driving image and the corresponding source image, and stores the modified source image data.

    LATENT DIFFUSION MODEL AUTODECODERS

    公开(公告)号:US20240395028A1

    公开(公告)日:2024-11-28

    申请号:US18400677

    申请日:2023-12-29

    Applicant: Snap Inc.

    Abstract: Described is a system for improving machine learning models. In some cases, the system improves such models by identifying an autoencoder for a latent diffusion machine learning model, the latent diffusion machine learning model is trained to receive text as input and output an image based on the received text. The system identifies a number of channels in a decoder of the autoencoder, the decoder being configured to receive latent features as input and output images. The system further identifies a performance characteristic of the decoder and changes the node topology of the decoder based on the performance characteristic to generate an updated decoder. The system retrains the latent diffusion machine learning model using the updated decoder by inputting latent features to the updated decoder, receiving an outputted image from the updated decoder, and updating one or more weights of the decoder based on an assessment of the outputted image.

Patent Agency Ranking