Abstract:
Systems, apparatus and methods in a mobile device to enable and disable a depth sensor for tracking pose of the mobile device are presented. A mobile device relaying on a camera without a depth sensor may provide inadequate pose estimates, for example, in low light situations. A mobile device with a depth sensor uses substantial power when the depth sensor is enabled. Embodiments described herein enable a depth sensor only when images are expected to be inadequate, for example, accelerating or moving too fast, when inertial sensor measurements are too noisy, light levels are too low or high, an image is too blurry, or a rate of images is too slow. By only using a depth sensor when images are expected to be inadequate, battery power in the mobile device may be conserved and pose estimations may still be maintained.
Abstract:
Techniques are disclosed for estimating one or more parameters in a system. A device obtains measurements corresponding to a first set of features and a second set of features. The device estimates the parameters using an extended Kalman filter based on the measurements corresponding to the first set of features and the second set of features. The measurements corresponding to the first set of features are used to update the one or more parameters, and information corresponding to the first set of features. The measurements corresponding to the second set of features are used to update the parameters and uncertainty corresponding to the parameter. In on example, information corresponding to the second set of features is not updated during the estimating. Moreover, the parameters are estimated without projecting the information corresponding to the second set of features into a null-space.
Abstract:
Vision based tracking of a mobile device is used to remotely control a robot. For example, images captured by a mobile device, e.g., in a video stream, are used for vision based tracking of the pose of the mobile device with respect to the imaged environment. Changes in the pose of the mobile device, i.e., the trajectory of the mobile device, are determined and converted to a desired motion of a robot that is remote from the mobile device. The robot is then controlled to move with the desired motion. The trajectory of the mobile device is converted to the desired motion of the robot using a transformation generated by inverting a hand-eye calibration transformation.
Abstract:
Systems, apparatus and methods for triggering a depth sensor and/or limiting bandwidth and/or maintaining privacy are presented. By limiting use of a depth sensor to times when an optical image alone is insufficient, mobile device power is saved. Furthermore, by reducing a size of an optical image to only the portion of the image needed to detect an object, bandwidth is saved and privacy is maintained by not communicating unneeded or undesired information.
Abstract:
A Visual Inertial Tracker (VIT), such as a Simultaneous Localization And Mapping (SLAM) system based on an Extended Kalman Filter (EKF) framework (EKF-SLAM) can provide drift correction in calculations of a pose (translation and orientation) of a mobile device by obtaining location information regarding a target, obtaining an image of the target, estimating, from the image of the target, measurements relating to a pose of the mobile device based on the image and location information, and correcting a pose determination of the mobile device using an EKF, based, at least in part, on the measurements relating to the pose of the mobile device.
Abstract:
An accelerometer located within a mobile device is used to estimate a gravity vector on a target plane in a world coordinate system. The accelerometer makes multiple measurements, each measurement being taken when the mobile device is held stationary on the target plane and a surface of the mobile device faces and is in contact with a planar portion of the target plane. An average of the measurements is calculated. A rotational transformation between an accelerometer coordinate system and a mobile devices coordinate system is retrieved from a memory in the mobile device, where the mobile devices coordinate system is aligned with the surface of the mobile device. The rotational transformation is applied to the averaged measurements to obtain an estimated gravity vector in a world coordinate system defined by the target plane.
Abstract:
A mobile device tracks a relative pose between a camera and a target using Vision aided Inertial Navigation System (VINS), that includes a contribution from inertial sensor measurements and a contribution from vision based measurements. When the mobile device detects movement of the target, the contribution from the inertial sensor measurements to track the relative pose between the camera and the target is reduced or eliminated. Movement of the target may be detected by comparing vision only measurements from captured images and inertia based measurements to determine if a discrepancy exists indicating that the target has moved. Additionally or alternatively, movement of the target may be detected using projections of feature vectors extracted from captured images.
Abstract:
Embodiments disclosed pertain to the use of user equipment (UE) for the generation of a 3D exterior envelope of a structure based on captured images and a measurement set associated with each captured image. In some embodiments, a sequence of exterior images of a structure is captured and a corresponding measurement set comprising Inertial Measurement Unit (IMU) measurements, wireless measurements (including Global Navigation Satellite (GNSS) measurements) and/or other non-wireless sensor measurements may be obtained concurrently. A closed-loop trajectory of the UE in global coordinates may be determined and a 3D structural envelope of the structure may be obtained based on the closed loop trajectory and feature points in a subset of images selected from the sequence of exterior images of the structure.
Abstract:
Embodiments disclosed obtain a plurality of measurement sets from a plurality of sensors in conjunction with the capture of a sequence of exterior and interior images of a structure while traversing locations in and around the structure. Each measurement set may be associated with at least one image. An external structural envelope of the structure is determined from exterior images of the structure and the corresponding outdoor trajectory of a UE. The position and orientation of the structure and the structural envelope is determined in absolute coordinates. Further, an indoor map of the structure in absolute coordinates may be obtained based on interior images of the structure, a structural envelope in absolute coordinates, and measurements associated with the indoor trajectory of the UE during traversal of the indoor area to capture the interior images.
Abstract:
Exemplary methods, apparatuses, and systems infer a context of a user or device. A computer vision parameter is configured according to the inferred context. Performing a computer vision task, in accordance with the configured computer vision parameter. The computer vision task may by at least one of: a visual mapping of an environment of the device, a visual localization of the device or an object within the environment of the device, or a visual tracking of the device within the environment of the device.