A method of operating a building management system is disclosed. The method includes determining, by a processing circuit, policy rankings for a plurality of control policies based on building operation data of a first previous time period, selecting, by the processing circuit, a set of control policies from among the plurality of control policies based on the policy rankings of the set of control policies satisfying a ranking threshold, generating, by the processing circuit, a plurality of prediction models for the set of control policies, selecting, by the processing circuit, a first prediction model of the plurality of prediction models based on building operation data of a second previous time period, and responsive to selecting the first prediction model, operating, by the processing circuit, the building management system using the first prediction model.