A complete end-to-end modeling system is provided that includes data sampling, feature engineering, action labeling, and model learning or learning from models built based on collected data. The end-to-end modeling process is performed via an automatic mechanism with minimal or reduced human intervention. A processor-readable medium is disclosed, storing processor-executable instructions to instantiate an automated data sampling and prediction structure training component, the automated data sampling and prediction structure training component being configured to automatically collect user event data samples, and use the collected user event data samples to train multiple prediction structures in parallel.