Text may be modified according to a specified attribute value, such as changing a 1-star review to a 5-star review. To modify the text, an input sequence of tokens may be obtained corresponding to the text. Mask scores may be computed for the tokens by processing the input sequence of tokens with masking neural network. One or more tokens may be selected using the mask scores, and a masked sequence of tokens may be generated by replacing the selected tokens with a mask token. The masked sequence of tokens may be processed by a language model neural network to select a replacement token for each of the mask tokens. Modified text may then be generated using the selected replacement tokens. The modified text may be used for any appropriate application, such as suggesting messages to users participating in a conversation.