Methods, systems, and apparatus, including computer programs encoded on one or more computer storage devices, for training and retraining predictive models. A series of training data sets for predictive modeling can be received, e.g., over a network from a client computing system. The training data included in the training data sets is different from initial training data that was used with multiple training functions to train multiple trained predictive models stored in a predictive model repository. The series of training data sets are used with multiple trained updateable predictive models obtained from the predictive model repository and multiple training functions to generate multiple retrained predictive models. An effectiveness score is generated for each of the retrained predictive models. A first trained predictive model is selected from among the trained predictive models included in the predictive model repository and the retrained predictive models based on their respective effectiveness scores.