公开(公告)号：US20220058773A1

公开(公告)日：2022-02-24

申请号：US17000260

申请日：2020-08-21

Applicant: Apple Inc.

Inventor： Jim C. Chou ,  Yun Gong

IPC: G06T5/00 ,  G06T3/40 ,  G06T7/11 ,  G06T5/50

Abstract: Methods and systems include neural network-based image processing and blending circuitry to blend an output of the neural network to compensate for potential artifacts from the neural network-based image processing. The neural network(s) apply image processing to image data using one or more neural networks as processed data. Enhance circuitry enhances the image data in a scaling circuitry to generate enhanced data. Blending circuitry receives the processed image data and the enhanced data along with an image plane of the processed data. The blending circuitry also determines whether the image processing using the one or more neural networks has applied a change to the image data greater than a threshold amount. The blending circuitry then, based at least in part in response to the change being greater than the threshold amount and/or edge information of the image data, blends the processed data with the enhanced data.

公开(公告)号：US11076176B2

公开(公告)日：2021-07-27

申请号：US16711189

申请日：2019-12-11

Applicant: Apple Inc.

Inventor： Jim C. Chou ,  Syed Muhammad A. Rizvi

IPC: H04N19/91 ,  H04N19/156 ,  H04N19/174 ,  H04N19/436 ,  H04N19/127

Abstract: Systems and methods for improving determination of encoded image data using a video encoding pipeline, which includes a first transcode engine that entropy encodes a first portion of a bin stream to determine a first bit stream including first encoded image data that indicates a first coding group row and that determines first characteristic data corresponding to the first bit stream to facilitate communicating a combined bit stream; and a second transcode engine that entropy encode a second portion of the bin stream to determine a second bit stream including second encoded image data that indicates a second coding group row while the first transcode engine entropy encodes the first portion of the bin stream and that determines second characteristic data corresponding to the second bit stream to facilitate communicating the combined bit stream, which includes the first bit stream and the second bit stream, to a decoding device.

公开(公告)号：US11043018B2

公开(公告)日：2021-06-22

申请号：US16662952

申请日：2019-10-24

Applicant: Apple Inc.

Inventor： Arthur Y Zhang ,  Ray L. Chang ,  Timothy R. Oriol ,  Ling Su ,  Gurjeet S. Saund ,  Guy Cote ,  Jim C. Chou ,  Hao Pan ,  Tobias Eble ,  Avi Bar-Zeev ,  Sheng Zhang ,  Justin A. Hensley ,  Geoffrey Stahl

IPC: G06F3/01 ,  G06T15/00 ,  H04W76/10 ,  G06T3/00 ,  G06T9/00 ,  H04W88/08

Abstract: A mixed reality system that includes a device and a base station that communicate via a wireless connection The device may include sensors that collect information about the user's environment and about the user. The information collected by the sensors may be transmitted to the base station via the wireless connection. The base station renders frames or slices based at least in part on the sensor information received from the device, encodes the frames or slices, and transmits the compressed frames or slices to the device for decoding and display. The base station may provide more computing power than conventional stand-alone systems, and the wireless connection does not tether the device to the base station as in conventional tethered systems. The system may implement methods and apparatus to maintain a target frame rate through the wireless link and to minimize latency in frame rendering, transmittal, and display.

公开(公告)号：US20210127117A1

公开(公告)日：2021-04-29

申请号：US17252701

申请日：2019-06-27

Applicant: Apple Inc.

Inventor： Hao Pan ,  Jim C. Chou ,  Felix C. Fernandes

IPC: H04N19/146 ,  H04N19/14 ,  H04N19/63 ,  H04N19/169 ,  H04N19/176 ,  H04L29/06

Abstract: A video encoding system in which pixel data is decomposed into frequency bands prior to encoding. The frequency bands for a slice of a frame may be buffered so that complexity statistics may be calculated across the frequency bands prior to encoding. The statistics may then be used by a rate control component in determining quantization parameters for the frequency bands for modulating the rate in the encoder for the current slice. The quantization parameters for the frequency bands may be calculated jointly to optimize the quality of the displayed frames after decoder reconstruction and wavelet synthesis on a receiving device. Information about one or more previously processed frames may be used in combination with the statistics for a current slice in determining the quantization parameters for the current slice.

公开(公告)号：US20200099942A1

公开(公告)日：2020-03-26

申请号：US16583153

申请日：2019-09-25

Applicant: Apple Inc.

Inventor： Alexandros Tourapis ,  Yeping Su ,  Khaled Mammou ,  Jungsun Kim ,  David W. Singer ,  Fabrice A. Robinet ,  Guy Cote ,  Jim C. Chou ,  Hsi-Jung Wu

IPC: H04N19/186 ,  H04N19/124 ,  H04N19/176 ,  H04N19/159

Abstract: Support for additional components may be specified in a coding scheme for image data. A layer of a coding scheme that specifies color components may also specify additional components. Characteristics of the components may be specified in the same layer or a different layer of the coding scheme. An encoder or decoder may identify the specified components and determine the respective characteristics to perform encoding and decoding of image data.

公开(公告)号：US20200082498A1

公开(公告)日：2020-03-12

申请号：US16681631

申请日：2019-11-12

Applicant: Apple Inc.

Inventor： Tobias Eble ,  Jim C. Chou ,  Jian Zhou ,  Moinul Khan ,  Hariprasad Puthukkootil Rajagopal

IPC: G06T3/00 ,  G06T5/00 ,  G02B27/01 ,  G06T7/593

Abstract: In various implementations, a method includes obtaining a first frame that is characterized by a first resolution associated with a first memory allocation. In some implementations, the method includes down-converting the first frame from the first resolution to a second resolution that is lower than the first resolution initially defining the first frame in order to produce a reference frame. In some implementations, the second resolution is associated with a second memory allocation that is less than a target memory allocation derived from the first memory allocation. In some implementations, the method includes storing the reference frame in a non-transitory memory. In some implementations, the method includes obtaining a second frame that is characterized by the first resolution. In some implementations, the method includes performing an error correction operation on the second frame based on the reference frame stored in the non-transitory memory.

公开(公告)号：US20200058152A1

公开(公告)日：2020-02-20

申请号：US16662952

申请日：2019-10-24

Applicant: Apple Inc.

Inventor： Arthur Y. Zhang ,  Ray L. Chang ,  Timothy R. Oriol ,  Ling Su ,  Gurjeet S. Saund ,  Guy Cote ,  Jim C. Chou ,  Hao Pan ,  Tobias Eble ,  Avi Bar-Zeev ,  Sheng Zhang ,  Justin A. Hensley ,  Geoffrey Stahl

IPC: G06T15/00 ,  G06T3/00 ,  G06T9/00 ,  G06F3/01 ,  H04W76/10

Abstract: A mixed reality system that includes a device and a base station that communicate via a wireless connection The device may include sensors that collect information about the user's environment and about the user. The information collected by the sensors may be transmitted to the base station via the wireless connection. The base station renders frames or slices based at least in part on the sensor information received from the device, encodes the frames or slices, and transmits the compressed frames or slices to the device for decoding and display. The base station may provide more computing power than conventional stand-alone systems, and the wireless connection does not tether the device to the base station as in conventional tethered systems. The system may implement methods and apparatus to maintain a target frame rate through the wireless link and to minimize latency in frame rendering, transmittal, and display.

公开(公告)号：US10477233B2

公开(公告)日：2019-11-12

申请号：US14871778

申请日：2015-09-30

Applicant: Apple Inc.

Inventor： Jim C. Chou ,  Mark P. Rygh ,  Guy Côté

IPC: H04N19/52 ,  H04N19/583 ,  H04N19/59 ,  H04N19/567 ,  H04N19/587 ,  H04N19/573 ,  H04N19/61 ,  H04N19/533 ,  H04N19/523 ,  H04N19/156 ,  H04N19/57

Abstract: System and method for improving operational efficiency of a video encoding pipeline used to encode image data. The video encoding pipeline includes a mode decision block, which selects a first inter-frame prediction mode used to prediction encode a first prediction unit, and a motion estimation block, which receives the first inter-frame prediction mode as feedback from the mode decision block when processing a second prediction unit; determines an initial candidate inter-frame prediction mode of the second prediction unit based at least in part on the first inter-frame prediction mode; and determines a final candidate inter-frame prediction mode of the second prediction unit by performing a first motion estimation search based at least in part on the initial candidate inter-frame prediction mode. The mode decision block determines a rate-distortion cost associated with the final candidate inter-frame prediction mode and a prediction mode used to prediction encode the second prediction unit based at least in part on the rate-distortion cost.

公开(公告)号：US10403192B2

公开(公告)日：2019-09-03

申请号：US15655591

申请日：2017-07-20

Applicant: APPLE INC.

Inventor： Marc Albrecht ,  Mahesh B. Chappalli ,  Christopher P. Tann ,  Jim C. Chou ,  Guy Cote

IPC: G09G3/20 ,  G09G3/32 ,  G09G3/36 ,  G09G3/3208

Abstract: Devices and methods for error diffusion and spatiotemporal dithering are provided. By way of example, a method of operating a display includes receiving a pixel input, a set of pixel coordinates, and a current frame number. A kernel and a particular kernel bit of the kernel is selected from a set of kernels, based upon the pixel input, the pixel coordinates, the frame number, or any combination thereof. A dithered output is determined based at least in part upon the kernel bit. When the display is in a diamond pixel configuration, the dithered output is applied in accordance with a diamond pattern formed by red, blue, or red and blue pixel channels.

公开(公告)号：US10225572B2

公开(公告)日：2019-03-05

申请号：US14871848

申请日：2015-09-30

Applicant: Apple Inc.

Inventor： Jim C. Chou ,  Mark P. Rygh ,  Guy Côté

IPC: H04N7/12 ,  H04N11/02 ,  H04N11/04 ,  H04N19/42 ,  H04N19/139 ,  H04N19/147 ,  H04N19/182 ,  H04N19/186 ,  H04N19/109 ,  H04N19/156 ,  H04N19/157 ,  H04N19/17

Abstract: System and method for improving operational efficiency of a video encoding pipeline used to encode image data. The video encoding pipeline includes a motion estimation setup block, which dynamically adjusts a setup configuration of the motion estimation block based at least in part on operational parameters of the video encoding pipeline and select an initial candidate inter-frame prediction mode based at least on the setup configuration, a full-pel motion estimation block, which determines an intermediate candidate inter-frame prediction mode by performing a motion estimation search based on the initial candidate inter-frame prediction mode, a sub-pel motion estimation block, which determines a final candidate inter-frame prediction by performing a motion estimation search based on the intermediate candidate inter-frame prediction mode, and a mode decision block, which determines a rate-distortion cost associated with the final candidate inter-frame prediction mode and determines a prediction mode used to prediction encoding the image data.