公开(公告)号：WO2017138994A1

公开(公告)日：2017-08-17

申请号：PCT/US2016/063103

申请日：2016-11-21

Applicant: QUALCOMM INCORPORATED

Inventor： NGUYEN, Phi Hung Le ,  BI, Ning

IPC: H04N13/00 ,  G06F3/01 ,  G06T3/00 ,  G06T5/00 ,  G06T15/00 ,  H04N13/02

CPC classification number: G06T3/4092 ,  G06F3/01 ,  G06T3/00 ,  G06T3/0025 ,  G06T3/0087 ,  G06T15/00 ,  G06T19/006 ,  H04N13/106 ,  H04N13/279 ,  H04N13/383

Abstract: Techniques are described for generating and rendering video content based on area of interest (also referred to as foveated rendering) to allow 360 video or virtual reality to be rendered with relatively high pixel resolution even on hardware not specifically designed to render at such high pixel resolution. Processing circuitry may be configured to keep the pixel resolution within a first portion of an image of one view at the relatively high pixel resolution, but reduce the pixel resolution through the remaining portions of the image of the view based on an eccentricity map and/or user eye placement. A device may receive the images of these views and process the images to generate viewable content (e.g., perform stereoscopic rendering or interpolation between views). Processing circuitry may also make use of future frames within a video stream and base predictions on those future frames.

Abstract translation: 描述了用于基于感兴趣的区域（也称为foveated rendering）来生成和呈现视频内容的技术，以允许即使在硬件上并非专门地以相对高的像素分辨率呈现360视频或虚拟现实 旨在呈现如此高的像素分辨率。 处理电路可以被配置为将像素分辨率保持在相对高的像素分辨率下的一个视图的图像的第一部分内，但是基于偏心率图来减小通过视图的图像的剩余部分的像素分辨率和/或 用户眼睛放置。 设备可以接收这些视图的图像并处理图像以生成可观看的内容（例如，在视图之间执行立体渲染或内插）。 处理电路还可以利用视频流内的未来帧，并对这些未来帧进行基础预测。

公开(公告)号：WO2017065883A1

公开(公告)日：2017-04-20

申请号：PCT/US2016/049072

申请日：2016-08-26

Applicant: QUALCOMM INCORPORATED

Inventor： GUO, Feng ,  BI, Ning ,  FORUTANPOUR, Bijan ,  ZHANG, Tao ,  REN, Jianfeng

IPC: G06T3/00 ,  G06T7/00

CPC classification number: G06T19/20 ,  G02B27/0172 ,  G02B2027/0138 ,  G02B2027/014 ,  G02B2027/0147 ,  G06T3/00 ,  G06T7/30 ,  G06T19/006 ,  G06T2200/32 ,  G06T2207/10028 ,  G06T2207/20221 ,  H04N7/183

Abstract: A method performed by an electronic device is described. The method includes obtaining a combined image. The combined image includes a combination of images captured from one or more image sensors. The method also includes obtaining depth information. The depth information is based on a distance measurement between a depth sensor and at least one object in the combined image. The method further includes adjusting a combined image visualization based on the depth information.

Abstract translation: 描述了由电子设备执行的方法。 该方法包括获得组合图像。 组合图像包括从一个或多个图像传感器捕获的图像的组合。 该方法还包括获得深度信息。 深度信息基于深度传感器与组合图像中的至少一个物体之间的距离测量值。 该方法还包括基于深度信息调整组合图像可视化。

公开(公告)号：WO2017062377A1

公开(公告)日：2017-04-13

申请号：PCT/US2016/055364

申请日：2016-10-04

Applicant: QUALCOMM INCORPORATED

Inventor： CHEN, Ying ,  ZHANG, Xuerui ,  TIWARI, Mayank ,  BI, Ning ,  PANDA, Prasanjit

IPC: H04N19/105 ,  H04N19/176 ,  H04N19/70 ,  H04N19/46 ,  H04N19/162 ,  H04N19/58

CPC classification number: H04N19/159 ,  H04N19/105 ,  H04N19/162 ,  H04N19/172 ,  H04N19/176 ,  H04N19/188 ,  H04N19/46 ,  H04N19/58 ,  H04N19/70

Abstract: Techniques and systems are provided for performing predictive random access using a background picture. For example, a method of decoding video data includes obtaining an encoded video bitstream comprising a plurality of pictures. The plurality of pictures include a plurality of predictive random access pictures. A predictive random access picture is at least partially encoded using inter-prediction based on at least one background picture. The method further includes determining, for a time instance of the video bitstream, a predictive random access picture of the plurality of predictive random access pictures with a time stamp closest in time to the time instance. The method further includes determining a background picture associated with the predictive random access picture, and decoding at least a portion of the predictive random access picture using inter-prediction based on the background picture.

Abstract translation: 提供技术和系统用于使用背景图片执行预测性随机访问。 例如，解码视频数据的方法包括获得包括多个图像的编码视频比特流。 多个图像包括多个预测性随机存取图像。 使用基于至少一个背景图片的帧间预测，至少部分地对预测随机存取图像进行编码。 所述方法还包括：对于所述视频比特流的时间实例，确定所述多个预测性随机访问图像的预测随机访问图像具有与时间实例在时间上最接近的时间戳。 该方法还包括确定与预测随机接入图像相关联的背景图像，以及使用基于背景图像的帧间预测来对至少一部分预测随机接入图像进行解码。

公开(公告)号：WO2017049589A1

公开(公告)日：2017-03-30

申请号：PCT/CN2015/090733

申请日：2015-09-25

Applicant: QUALCOMM INCORPORATED ,  BI, Ning ,  LING, Fan ,  DING, Jingting ,  SUN, Yu

Inventor： BI, Ning ,  LING, Fan ,  DING, Jingting ,  SUN, Yu

IPC: H04N7/18

CPC classification number: G06K9/00751 ,  G06F17/30843

Abstract: A system and method of video processing are disclosed. In a particular implementation, a device includes a frame buffer configured to receive a video frame sequence and to store the video frame sequence and a frame index for each frame of the video frame sequence. The device also includes a processor configured to compress a summary frame into an encoded file representative of the summary frame. The summary frame includes a first representation of an object at a first position in a first video frame associated with a first frame index of the video frame sequence. The summary frame further includes a second representation of an object at a second position in a second video frame associated with a second frame index associated with the video frame sequence. The device also includes a transmitter configured to output a notification signal associated with the summary frame.

Abstract translation: 公开了一种视频处理的系统和方法。 在具体实现中，设备包括帧缓冲器，其被配置为接收视频帧序列并且存储视频帧序列和帧视频帧序列的每个帧的帧索引。 该设备还包括被配置为将汇总帧压缩成表示汇总帧的编码文件的处理器。 总结帧包括在与视频帧序列的第一帧索引相关联的第一视频帧中的第一位置处的对象的第一表示。 摘要帧还包括在与与视频帧序列相关联的第二帧索引相关联的第二视频帧中的第二位置处的对象的第二表示。 该设备还包括被配置为输出与摘要帧相关联的通知信号的发射机。

公开(公告)号：WO2017049577A1

公开(公告)日：2017-03-30

申请号：PCT/CN2015/090684

申请日：2015-09-25

Applicant: QUALCOMM INCORPORATED ,  BI, Ning ,  LING, Fan ,  DING, Jingting ,  SUN, Yu

Inventor： BI, Ning ,  LING, Fan ,  DING, Jingting ,  SUN, Yu

IPC: H04N21/8549 ,  G06F17/30

CPC classification number: H04N21/8549 ,  G06F17/3079 ,  G06F17/30858 ,  H04N21/2187 ,  H04N21/23418 ,  H04N21/234318 ,  H04N21/8153 ,  H04N21/8453 ,  H04N21/8455

Abstract: A system and method of video processing are disclosed. In a particular implementation, a device includes a processor configured to generate index data for video content. The index data includes a summary frame and metadata. The summary frame is associated with a portion of the video content and illustrates multiple representations of an object included in the portion of the video content. The metadata includes marker data that indicates a playback position of the video content. The playback position is associated with the summary frame. The device also includes a memory configured to store the index data.

Abstract translation: 公开了一种视频处理的系统和方法。 在特定实现中，设备包括被配置为生成用于视频内容的索引数据的处理器。 索引数据包括摘要帧和元数据。 摘要帧与视频内容的一部分相关联，并且示出了包括在视频内容的该部分中的对象的多个表示。 元数据包括指示视频内容的播放位置的标记数据。 播放位置与摘要帧相关联。 该设备还包括被配置为存储索引数据的存储器。

公开(公告)号：WO2016195792A1

公开(公告)日：2016-12-08

申请号：PCT/US2016/025019

申请日：2016-03-30

Applicant: QUALCOMM INCORPORATED

Inventor： ZHANG, Tao ,  GUO, Feng ,  BI, Ning ,  FORUTANPOUR, Bijan ,  NGUYEN, Phi Hung

IPC: G06T15/20 ,  G06T3/00

CPC classification number: H04N5/23238 ,  B60R1/00 ,  B60R2300/105 ,  B60R2300/303 ,  G06T3/0062 ,  G06T15/205 ,  H04N5/23261 ,  H04N5/265 ,  H04N13/239

Abstract: An electronic device is described. The electronic device includes a processor. The processor is configured to obtain images from a plurality of cameras. The processor is also configured to project each image to a respective 3-dimensional (3D) shape for each camera. The processor is further configured to generate a combined view from the images.

Abstract translation: 描述电子设备。 电子设备包括处理器。 处理器被配置为从多个摄像机获得图像。 处理器还被配置为将每个图像投影到每个相机的相应的三维（3D）形状。 处理器还被配置为从图像生成组合视图。

公开(公告)号：WO2015123792A1

公开(公告)日：2015-08-27

申请号：PCT/CN2014/000172

申请日：2014-02-24

Applicant: QUALCOMM INCORPORATED ,  ZHANG, Lei ,  BI, Ning ,  LING, Fan

Inventor： ZHANG, Lei ,  BI, Ning ,  LING, Fan

IPC: G06F17/24

CPC classification number: G06T11/60 ,  G06F3/0488 ,  G06K9/2081 ,  G06K9/342 ,  G06T7/11 ,  G06T7/194 ,  G06T2207/10004

Abstract: A method of manipulating an image by a device is disclosed. The method includes segmenting image data corresponding to the image into a first image layer and a second image layer. The method further includes adjusting a first attribute of the first image layer independently of a second attribute of the second image layer based on user input.

Abstract translation: 公开了一种通过设备操纵图像的方法。 该方法包括将对应于图像的图像数据分割成第一图像层和第二图像层。 该方法还包括基于用户输入来独立于第二图像层的第二属性调整第一图像层的第一属性。

公开(公告)号：WO2014109851A1

公开(公告)日：2014-07-17

申请号：PCT/US2013/074072

申请日：2013-12-10

Applicant: QUALCOMM INCORPORATED

Inventor： BI, Ning ,  ZHANG, Xuerui

IPC: H04N13/02 ,  G06T15/00

CPC classification number: H04N13/279 ,  G06T15/005

Abstract: The example techniques of this disclosure are directed to generating a stereoscopic view from an application designed to generate a mono view. For example, the techniques may modify instructions for a vertex shader based on a viewing angle. When the modified vertex shader is executed, the modified vertex shader may generate coordinates for vertices for a stereoscopic view based on the viewing angle.

Abstract translation: 本公开的示例性技术涉及从被设计成生成单视图的应用产生立体视图。 例如，这些技术可以基于视角修改顶点着色器的指令。 当修改的顶点着色器被执行时，修改的顶点着色器可以基于视角为立体视图的顶点生成坐标。

公开(公告)号：WO2014042795A2

公开(公告)日：2014-03-20

申请号：PCT/US2013/054423

申请日：2013-08-09

Applicant: QUALCOMM INCORPORATED

Inventor： YANG, Ruiduo ,  BI, Ning

IPC: G06T7/20

CPC classification number: H04N13/204 ,  G06T7/194 ,  G06T7/254 ,  G06T7/50 ,  G06T7/73 ,  G06T19/006 ,  G06T2207/10028

Abstract: In a particular embodiment, a method includes evaluating, at a mobile device, a first area of pixels to generate a first result. The method further includes evaluating, at the mobile device, a second area of pixels to generate a second result. Based on comparing a threshold with a difference between the first result and the second result, a determination is made that the second area of pixels corresponds to a background portion of a scene or a foreground portion of the scene.

Abstract translation: 在特定实施例中，一种方法包括在移动设备处评估第一区域的像素以产生第一结果。 该方法还包括在移动设备处评估第二区域的像素以产生第二结果。 基于将阈值与第一结果与第二结果之间的差进行比较，确定第二区域像素对应于场景的背景部分或场景的前景部分。

公开(公告)号：WO2012074937A1

公开(公告)日：2012-06-07

申请号：PCT/US2011/062261

申请日：2011-11-28

Applicant: QUALCOMM INCORPORATED ,  ZHANG, Xuerui ,  BI, Ning ,  QI, Yingyong

Inventor： ZHANG, Xuerui ,  BI, Ning ,  QI, Yingyong

IPC: H04N13/00 ,  H04N13/02 ,  G06T19/00

CPC classification number: G06T19/006 ,  H04N13/156

Abstract: A three dimensional (3D) mixed reality system combines a real 3D image or video, captured by a 3D camera for example, with a virtual 3D image rendered by a computer or other machine to render a 3D mixed-reality image or video. A 3D camera can acquire two separate images (a left and a right) of a common scene, and superimpose the two separate images to create a real image with a 3D depth effect. The 3D mixed-reality system can determine a distance to a zero disparity plane for the real 3D image, determine one or more parameters for a projection matrix based on the distance to the zero disparity plane, render a virtual 3D object based on the projection matrix, combine the real image and the virtual 3D object to generate a mixed-reality 3D image.

Abstract translation: 三维（3D）混合现实系统将由3D摄像机捕获的真实3D图像或视频与由计算机或其他机器呈现的虚拟3D图像组合以渲染3D混合现实图像或视频。 3D摄像机可以获取公共场景的两个单独的图像（左和右），并且叠加两个分离的图像以创建具有3D深度效果的实际图像。 3D混合现实系统可以确定实际3D图像到零视差平面的距离，基于到零视差平面的距离确定投影矩阵的一个或多个参数，基于投影矩阵渲染虚拟3D对象 ，组合真实图像和虚拟3D对象以产生混合现实的3D图像。