An imager utilizes a division of focal plane polarization and color camera to measure motion, depth and orientation of objects in a scene in real-time. In various examples, structured light, polarization-controlled discrete reflectors, and/or spatially varying discrete light sources are used to provide light of controlled polarization and color from an object in a scene to a camera. The camera utilizes a pixelated optical filter with a pattern of varying polarization filters across the pixel array, and optionally an integrated color filter pattern. Light measurements are processed to determine polarization state of light received from the object, whence orientation, position, and/or other properties of the object are determined. Systems are operable with a single camera. Applications include virtual reality, gaming, robotics, autonomous vehicles, tele-surgery, industrial automation, 3-D scanning, surveillance, and remote interaction.