System, methods, and other embodiments described herein relate to end-user modification of the driving behavior of a vehicle when operating in an autonomous driving mode. One embodiment stores disengagement data associated with an autonomous driving mode of the vehicle; outputs, to an end user from the stored disengagement data, a report of a disengagement that occurred along a particular route identified by the end user due to incorrect identification, by a sensor system of the vehicle, of an object; receiving, from the end user, a corrective input that includes labeling the object; and modifying automatically one or more autonomous driving modules of the vehicle in accordance with the corrective input to prevent future disengagements when the vehicle is traveling along the particular route in the autonomous driving mode.