In one or more aspects of the present disclosure, a method, a computer program product and a system for reconstructing scene features of an object in 3D space using structure-from-motion feature-tracking includes acquiring a first camera frame at a first camera position; extracting image features from the first camera frame; initializing a first set of 3D points from the extracted image features; acquiring a second camera frame at a second camera position; predicting a second set of 3D points by converting their positions and variances to the second camera position; projecting the predicted 3D positions to an image plane of the second camera to obtain 2D predictions of the image features; measuring an innovation of the predicted 2D image features; and updating estimates of 3D points based on the measured innovation to reconstruct scene features of the object image in 3D space.