Methods, systems, and apparatus, including computer programs encoded on computer storage media for dialogue systems. A transcription of a user utterance is obtained. The transcription of the utterance is tokenized to identify multiple tokens for the utterance. Token-level utterance encodings corresponding to different tokens of the transcription are generated. A system action encoding from data indicating system actions previously performed by the dialogue system are generated. A dialogue context vector based on the utterance encoding and the system action encoding are generated. The token-level utterance encodings, the system action encoding, and the dialogue context vector are processed using a slot tagger to produce token-level output vectors. A limited set of candidate token classifications for the tokens of the user utterance are determined based on the token-level utterance encodings. A response for output is provided in response to the user utterance.