The presentation of an automated assistant response may be selectively pre-empted in response to a hot-word free utterance that is received during the presentation and that is determined to be likely directed to the automated assistant. The determination that the utterance is likely directed to the automated assistant may be performed, for example, using an utterance classification operation that is performed on audio data received during presentation of the response, and based upon such a determination, the response may be pre-empted with another response associated with the later-received utterance. In addition, the duration that is used to determine when a session should be terminated at the conclusion of a conversation between a user and an automated assistant may be dynamically controlled based upon when the presentation of a response has completed.