Control commands in a system and method for autonomously driving a vehicle or partially autonomously driving a vehicle are generated on the basis of an environment representation, which is generated from sensor signal of a sensing means. The environment representation where ambiguous objects are identified is such that information obtained from the driver of the vehicle are added. The system generates an information request. Additional information is extracted and accumulated in the environment representation map. Traffic is then determined and suitable control signals for the vehicles are generated. In case no ambiguous objects are included in the environment representation but the system is not capable of deciding on traffic, the driver is asked to disambiguate the situation or to instruct on dealing with the traffic.