A trailer assist system for a vehicle includes a camera disposed at a rear portion of a vehicle and viewing a trailer hitch disposed at the vehicle. A control includes an image processor that processes image data captured by the camera that is representative of images of the trailer hitch viewed by the camera. The control, responsive to processing at the control of image data captured by the camera, detects a feature of the trailer hitch at determines Cartesian coordinates of the trailer hitch location and transforms the Cartesian coordinates of the detected feature to cylindrical coordinates. The control, responsive at least in part to determining the cylindrical coordinates of the detected feature, determines a three-dimensional location of the trailer hitch at the vehicle.