A driver assistance system for a vehicle includes an imager and a control having an image processor. The imager is disposed at or proximate to an in-cabin portion of a windshield of a vehicle and has a forward field of view to the exterior of the vehicle through the vehicle windshield. The driver assistance system identifies objects in the forward field of view of the imager via processing of captured image data by the image processor. At least in part responsive to processing of captured image data by the image processor, a determination is made that the vehicle is being operated in an urban environment. Responsive to the determination that the vehicle is being operated in an urban environment, operation of a driver assistance function of the vehicle is inhibited.