In various examples, a trigger signal may be received that is indicative of a vehicle maneuver to be performed by a vehicle. A recommended vehicle trajectory for the vehicle maneuver may be determined in response to the trigger signal being received. To determine the recommended vehicle trajectory, sensor data may be received that represents a field of view of at least one sensor of the vehicle. A value of a control input and the sensor data may then be applied to a machine learning model(s) and the machine learning model(s) may compute output data that includes vehicle control data that represents the recommended vehicle trajectory for the vehicle through at least a portion of the vehicle maneuver. The vehicle control data may then be sent to a control component of the vehicle to cause the vehicle to be controlled according to the vehicle control data.