LOW POWER CAMERA PIPELINE FOR COMPUTER VISION MODE IN AUGMENTED REALITY EYEWEAR

    公开(公告)号:WO2022072261A1

    公开(公告)日:2022-04-07

    申请号:PCT/US2021/052136

    申请日:2021-09-27

    Applicant: SNAP INC.

    Abstract: An eyewear device having an image processor operable in a camera pipeline for computer vision (CV) and in augmented reality (AR) systems. The image processor is configured to selectively control a plurality of cameras to provide images having a first resolution in the high power AR mode, and to provide the images having a second resolution in the low power CV mode. The first resolution is higher than the second resolution, and the plurality of cameras consume less power in the CV mode than the AR mode. The image processor controls the camera pipeline to process the first resolution, high image quality (IQ) images from the plurality of cameras to operate in the AR mode, and controls the camera pipeline to process the second resolution, lower IQ images from the plurality of cameras to operate in the CV mode. Substantial power is saved by reducing the resolution of the images using downscaling in the cameras themselves in the CV mode.

    ROLLING SHUTTER CAMERA PIPELINE EXPOSURE TIMESTAMP ERROR DETERMINATION

    公开(公告)号:WO2022005734A1

    公开(公告)日:2022-01-06

    申请号:PCT/US2021/037317

    申请日:2021-06-15

    Applicant: SNAP INC.

    Abstract: Camera pipeline exposure timestamp error determination methods and systems that detect latency in rolling shutter camera systems. The exposure timestamp error determination system includes a rolling shutter image sensor configured to capture an image and a processor. The processor determines timestamp error by capturing the image using the rolling shutter image sensor where the image includes a bar code encoded with a barcode timestamp. The processor then obtains a system exposure timestamp corresponding to capture of the image by the rolling shutter image sensor and the barcode timestamp by decoding the bar code. Pipeline exposure timestamp error is then determined for the rolling shutter camera system by comparing the obtained barcode timestamp to the system exposure timestamp.

    ADAPTIVE BRIGHTNESS FOR AUGMENTED REALITY DISPLAY

    公开(公告)号:WO2022260954A1

    公开(公告)日:2022-12-15

    申请号:PCT/US2022/032175

    申请日:2022-06-03

    Applicant: SNAP INC.

    Abstract: Disclosed are systems and methods for adaptively adjusting brightness of a wearable device projection system. The systems and methods perform operations comprising: causing projection elements of the AR wearable device to project an image; receiving a measure of ambient light from an ambient light sensor; adjusting one or more hardware parameters of the projection elements of the AR wearable device based on the measure of ambient light; modifying one or more color values of the image displayed by the projection elements of the AR wearable device based on the measure of ambient light; and projecting the image with the modified color values using the projection elements of the AR wearable device with the adjusted one or more hardware parameters.

    MULTI-PURPOSE CAMERAS FOR AUGMENTED REALITY AND COMPUTER VISION APPLICATIONS

    公开(公告)号:WO2022072097A1

    公开(公告)日:2022-04-07

    申请号:PCT/US2021/048123

    申请日:2021-08-29

    Applicant: SNAP INC.

    Abstract: Eyewear having an image signal processor (ISP) dynamically operable in a camera pipeline for augmented reality (AR) and computer vision (CV) systems. Multi-purpose cameras are used for simultaneous image capture and CV on wearable AR devices. The cameras are coupled to a frame and configured to generate images, wherein the cameras and the ISP are configured to operate in a first AR mode and capture images having a first resolution suitable for use in AR, and are configured to operate in a second CV mode to provide the images having a second resolution suitable for use in CV. The first resolution in the AR mode is higher than the second resolution in the CV mode, and the cameras and the ISP consume less power in the second CV mode than the first AR mode. The cameras and the ISP save significant system power by operating in the low power mode CV mode.

Patent Agency Ranking