A vision system of a vehicle includes a camera configured to be disposed at a vehicle so as to have a field of view exterior of the vehicle. A tag or retroreflective element is disposed at another vehicle. With the camera disposed at the vehicle and when the tag element is present in the camera's field of view, an image processor is operable to process captured image data to determine a pattern of the tag element present in the field of view of the camera. The image processor compares the determined pattern of the tag element to a database of patterns and classifies the tag element at least in part responsive to determination that the determined pattern of the tag element generally matches a pattern of the database. The pattern of the tag element disposed at the other vehicle includes information pertaining to a type of the other vehicle.