A vehicular vision system includes a camera disposed at a vehicle and viewing exterior of the vehicle. A control has an image processor for processing frames of image data captured by the camera. The control receives vehicle motion data indicative of motion of the vehicle when the vehicle is moving. When the vehicle is moving, the control, responsive at least in part to image processing of multiple frames of captured image data, detects an object present exterior of the vehicle. The control, responsive at least in part to the received vehicle motion data and image processing of multiple frames of captured image data, determines motion of the detected object relative to the moving vehicle by (i) determining corresponding object points in at least two frames of captured image data and (ii) estimating object motion trajectory of the detected object based at least in part on the determined corresponding object points.