Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating responses using task-independent conversational systems are provided. In one example method, a response to a user text input is generated by updating a state of the conversation based on the user text input, generating a conversational (task-independent) output, and determining whether to provide a conversational response based on the conversational output, or to additionally generate a task-specific output.