A system for controlling heating, ventilation, or air conditioning (HVAC) equipment of a building includes one or more processing circuits configured to generate simulated building data using a simulation model of the building, pre-train a reinforcement learning (RL) model using the simulated building data, operate the HVAC equipment of the building using the RL model, and retrain the RL model using actual building data generated responsive to operating the HVAC equipment using the RL model.