A vehicular driver monitoring system includes an interior-viewing camera viewing at least a head region of a driver of a vehicle, and a plurality of exterior-viewing cameras disposed at and viewing exterior of the vehicle. Based at least in part on processing at an electronic control unit (ECU) of image data captured by the interior-viewing camera, the vehicular driver monitoring system monitors the driver of the vehicle. Based at least in part on processing at the ECU of image data captured by at least one exterior-viewing camera of the plurality of exterior-viewing cameras, the vehicular driver monitoring system estimates a current driving workload of the driver of the vehicle. Based at least in part on monitoring of the driver and the estimated current driving workload, the system classifies the driver as being at least one selected from the group consisting of (i) overloaded, (ii) attentive, (iii) distracted and (iv) drowsy.