公开(公告)号：US20150350686A1

公开(公告)日：2015-12-03

申请号：US14290304

申请日：2014-05-29

Applicant: Apple Inc.

Inventor： Xiaosong Zhou ,  Chris Y. Chung ,  David R. Conrad ,  Dazhong Zhang ,  Feng Yi ,  Hsi-Jung Wu ,  Jae Hoon Kim ,  Jiefu Zhai ,  Peikang Song ,  Yunfei Zheng

IPC: H04N19/85 ,  H04N19/115 ,  H04N19/517 ,  H04N19/51 ,  H04N19/172 ,  H04N19/176

CPC classification number: H04N19/42 ,  H04N19/103

Abstract: A method and system of using a pre-encoder to improve encoder efficiency. The encoder may conform to ITU-T H.265 and the pre-encoder may conform to ITU-T H. 264. The pre-encoder may receive source video data and provide information regarding various coding modes, candidate modes, and a selected mode for coding the source video data. In an embodiment, the encoder may directly use the mode selected by the pre-encoder. In another embodiment, the encoder may receive both the source video data and information regarding the various coding modes (e.g., motion information, macroblock size, intra prediction direction, rate-distortion cost, and block pixel statistics) to simplify and/or refine its mode decision process. For example, the information provided by the pre-encoder may indicate unlikely modes, which unlikely modes need not be tested by the encoder, thus saving power and time.

Abstract translation: 一种使用预编码器来提高编码器效率的方法和系统。 编码器可以符合ITU-T H.265标准，并且预编码器可以符合ITU-T H.264的要求。预编码器可以接收源视频数据并提供关于各种编码模式，候选模式和选择模式的信息 用于对源视频数据进行编码。 在一个实施例中，编码器可以直接使用由预编码器选择的模式。 在另一个实施例中，编码器可以接收源视频数据和关于各种编码模式的信息（例如，运动信息，宏块大小，帧内预测方向，速率失真成本和块像素统计），以简化和/或改进其 模式决策过程。 例如，预编码器提供的信息可能指示不太可能的模式，不可能的模式不需要被编码器测试，从而节省功率和时间。

公开(公告)号：US20150350654A1

公开(公告)日：2015-12-03

申请号：US14290432

申请日：2014-05-29

Applicant: Apple Inc.

Inventor： Chris Y. Chung ,  Hsi-Jung Wu ,  Xiaosong Zhou ,  Dazhong Zhang

IPC: H04N19/146 ,  H04N19/156 ,  H04N19/51 ,  H04N19/172 ,  H04N19/46 ,  H04N19/127 ,  H04N19/44

CPC classification number: H04N19/46 ,  H04N19/132 ,  H04N19/154 ,  H04N19/156 ,  H04N19/164 ,  H04N19/172 ,  H04N19/587

Abstract: During video coding, frame rate conversion (FRC) capabilities of a decoder may be estimated. Based on the estimated FRC capabilities, an encoder may select a frame rate for a video coding session and may alter a frame rate of source video to match the selected frame rate. Thereafter, the resultant video may be coded and output to a channel. By incorporating knowledge of a decoder's FRC capabilities as source video is being coded, an encoder may reduce the frame rate of source video opportunistically. Bandwidth that is conserved by avoiding coding of video data in excess of the selected frame rate may be directed to coding of the remaining video at a higher bitrate, which can lead to increased quality of the coding session as a whole.

Abstract translation: 在视频编码期间，可以估计解码器的帧速率转换（FRC）能力。 基于估计的FRC能力，编码器可以选择用于视频编码会话的帧速率，并且可以改变源视频的帧速率以匹配所选择的帧速率。 此后，所得到的视频可被编码并输出到频道。 当编码源视频时，通过结合解码器的FRC能力的知识，编码器可以机会地降低源视频的帧速率。 通过避免超过所选帧速率的视频数据的编码而节省的带宽可以以更高的比特率对剩余视频进行编码，这可以导致整个编码会话的质量提高。

公开(公告)号：US11553200B2

公开(公告)日：2023-01-10

申请号：US16871378

申请日：2020-05-11

Applicant: Apple Inc.

Inventor： Xing Wen ,  Dazhong Zhang ,  Peikang Song ,  Xiaosong Zhou ,  Sudeng Hu ,  Hsi-Jung Wu ,  Jae Hoon Kim

IPC: H04N19/51 ,  H04N19/44 ,  H04N19/65 ,  H04N19/176 ,  H04N19/91

Abstract: Techniques are disclosed for coding and decoding video data using object recognition and object modeling as a basis of coding and error recovery. A video decoder may decode coded video data received from a channel. The video decoder may perform object recognition on decoded video data obtained therefrom, and, when an object is recognized in the decoded video data, the video decoder may generate a model representing the recognized object. It may store data representing the model locally. The video decoder may communicate the model data to an encoder, which may form a basis of error mitigation and recovery. The video decoder also may monitor deviation patterns in the object model and associated patterns in audio content; if/when video decoding is suspended due to operational errors, the video decoder may generate simulated video data by analyzing audio data received during the suspension period and developing video data from the data model and deviation(s) associated with patterns detected from the audio data.

公开(公告)号：US11184415B2

公开(公告)日：2021-11-23

申请号：US16405864

申请日：2019-05-07

Applicant: Apple Inc.

Inventor： Christopher M. Garrido ,  Dazhong Zhang ,  Karthick Santhanam ,  Patrick Miauton ,  Xiaoxiao Zheng ,  Bess Chan ,  Peter Shiang ,  Sudeng Hu ,  Peikang Song ,  Xiaosong Zhou

IPC: H04L29/06 ,  H04L29/08 ,  H04N21/235 ,  H04N21/6583

Abstract: Techniques presented herein provide an improved relay user experience and improved management of scarce computing and network resources as the number of relay endpoints increases. A sourcing endpoint device may generate a media feed, such as video and/or audio feed, representing contribution from a conference participant. The sourcing endpoint device may generate a priority value for the media feed, and the priority value may be transmitted to other members of the relay along with the input feed. Priority values of the different relay participants may be used by other devices, for example, intermediate servers or receiving endpoint devices, to manage aspects of the relay. For example, a relay server may prune streams from select endpoint devices based on relative priority values received from those devices. Alternatively, receiving endpoint devices may alter presentation of received feeds based on their associated priority values.

公开(公告)号：US11102515B2

公开(公告)日：2021-08-24

申请号：US16890245

申请日：2020-06-02

Applicant: Apple Inc.

Inventor： Jiefu Zhai ,  Dazhong Zhang ,  Xiaosong Zhou ,  Chris Y. Chung ,  Hsi-Jung Wu ,  Peikang Song ,  David R. Conrad ,  Jae Hoon Kim ,  Yunfei Zheng

IPC: H04N19/82 ,  H04N19/186

Abstract: Chroma deblock filtering of reconstructed video samples may be performed to remove blockiness artifacts and reduce color artifacts without over-smoothing. In a first method, chroma deblocking may be performed for boundary samples of a smallest transform size, regardless of partitions and coding modes. In a second method, chroma deblocking may be performed when a boundary strength is greater than 0. In a third method, chroma deblocking may be performed regardless of boundary strengths. In a fourth method, the type of chroma deblocking to be performed may be signaled in a slice header by a flag. Furthermore, luma deblock filtering techniques may be applied to chroma deblock filtering.

公开(公告)号：US11025933B2

公开(公告)日：2021-06-01

申请号：US15585581

申请日：2017-05-03

Applicant: Apple Inc.

Inventor： Dazhong Zhang ,  Hang Yuan ,  Peikang Song ,  Jae Hoon Kim ,  Xing Wen ,  Sudeng Hu ,  Xiaosong Zhou ,  Chris Y. Chung ,  Hsi-Jung Wu

IPC: H04N19/423 ,  H04N19/463 ,  H04N21/44 ,  H04N21/6379 ,  H04N21/6373

Abstract: Techniques are disclosed for managing memory allocations when coding video data according to multiple codec configurations. According to these techniques, devices may negotiate parameters of a coding session that include parameters of a plurality of different codec configurations that may be used during the coding session. A device may estimate sizes of decoded picture buffers for each of the negotiated codec configurations and allocate in its memory a portion of memory sized according to a largest size of the estimated decoded picture buffers. Thereafter, the devices may exchange coded video data. The exchange may involve decoding coded data of reference pictures and storing the decoded reference pictures in the allocated memory. During the coding session, the devices may toggle among the different negotiated codec configurations. As they do, reallocations of memory may be avoided.

公开(公告)号：US10924747B2

公开(公告)日：2021-02-16

申请号：US15443342

申请日：2017-02-27

Applicant: Apple Inc.

Inventor： Jae Hoon Kim ,  Chris Y. Chung ,  Dazhong Zhang ,  Hang Yuan ,  Hsi-Jung Wu ,  Jiefu Zhai ,  Xiaosong Zhou

IPC: H04B1/66 ,  H04N19/176 ,  H04N19/105 ,  H04N19/112 ,  H04N19/174 ,  H04N19/593 ,  H04N19/597

Abstract: Techniques are disclosed for coding and decoding video captured as cube map images. According to these techniques, padded reference images are generated for use during predicting input data. A reference image is stored in a cube map format. A padded reference image is generated from the reference image in which image data of a first view contained in reference image is replicated and placed adjacent to a second view contained in the cube map image. When coding a pixel block of an input image, a prediction search may be performed between the input pixel block and content of the padded reference image. When the prediction search identifies a match, the pixel block may be coded with respect to matching data from the padded reference image. Presence of replicated data in the padded reference image is expected to increase the likelihood that adequate prediction matches will be identified for input pixel block data, which will increase overall efficiency of the video coding.

公开(公告)号：US20200304837A1

公开(公告)日：2020-09-24

申请号：US16895815

申请日：2020-06-08

Applicant: Apple Inc.

Inventor： Yunfei Zheng ,  Dazhong Zhang ,  Xiaosong Zhou ,  Chris Y. Chung ,  Hsi-Jung Wu

IPC: H04N19/70

Abstract: An encoding system may include a video source that captures video image, a video coder, and a controller to manage operation of the system. The video coder may encode the video image into encoded video data using a plurality of subgroup parameters corresponding to a plurality of subgroups of pixels within a group. The controller may set the subgroup parameters for at least one of the subgroups of pixels in the video coder, based upon at least one parameters corresponding to the group. A decoding system may decode the video data based upon the motion prediction parameters.

公开(公告)号：US10757445B2

公开(公告)日：2020-08-25

申请号：US16173105

申请日：2018-10-29

Applicant: Apple Inc.

Inventor： Alexandros Tourapis ,  David W. Singer ,  Haitao Guo ,  Hsi-Jung Wu ,  Sorin C. Cismas ,  Xiaohua Yang ,  Yeping Su ,  Dazhong Zhang

IPC: H04N19/593 ,  H04N19/156 ,  H04N19/44 ,  H04N19/176 ,  H04N19/182 ,  H04N19/174

Abstract: Methods are described for encoding and decoding blocks of image data using intra block copying (IBC). A source block for intra block copying is selected from a source region of a current image that is closer to the current block than a threshold, wherein the source region does not include a portion of the current image that is further from the current block than the threshold.

公开(公告)号：US10652578B2

公开(公告)日：2020-05-12

申请号：US15888559

申请日：2018-02-05

Applicant: Apple Inc.

Inventor： Jae Hoon Kim ,  Dazhong Zhang ,  Hang Yuan ,  Jiefu Zhai ,  Ming Chen ,  Xiaosong Zhou ,  Chris Y. Chung ,  Hsi-Jung Wu

IPC: H04N19/597 ,  H04N19/70 ,  H04N19/105 ,  H04N19/139 ,  H04N19/162 ,  H04N19/167 ,  H04N19/176 ,  H04N19/52

Abstract: Image processing techniques may accelerate coding of viewport data contained within multi-view image data. According to such techniques, an encoder may shifting content of a multi-directional image data according to the viewport location data provided by a decoder. The encoder may code the shifted multi-directional image data by predictive coding, and transmit to the decoder, the coded multi-directional image data and data identifying an amount of the shift. Doing so may move the viewport location to positions in the image data that are coded earlier than the positions that the viewport location naturally occupies and, thereby, may accelerate coding. On decode, a decoder may compare its present viewport location with viewport location data provided by the encoder with coded video data. The decoder may decode the coded video data and extract a portion of the decoded video data corresponding to a present viewport location for display.