A first text corpus comprising punctuated and capitalized text is received. The words in the first text corpus are then annotated with a set of labels indicating a punctuation and a capitalization of each word. At an initial training stage, a machine learning model is trained on a first training set using the annotated words from the first text corpus and the labels. A second text corpus is received representing conversational speech. The words in the second text corpus are then annotated with the set of labels. In a re-training stage, the machine learning model is re-trained on a second training set comprising the annotated words from the second text corpus, and the labels. At an inference stage, the trained machine learning model is applied to a target set of words representing conversational speech to predict a punctuation and capitalization of each word in the target set.