Methods, systems, and devices are disclosed for managing building power, in one aspect, a method for managing building power includes determining values for power usage of a heating, ventilation, or air conditioning (HVAC) system in one or more zones of a building, the values including a cost of power value, a comfort value, a weighting function between the cost of power value and the comfort value, or a thermal storage value, in which the determining the values is based on a plurality of parameters including a price of power, a time of use, a total power allocation, or random variables including weather and building occupancy factors, and determining a power level for a plurality of states based on the determined values, the plurality of states corresponding to different levels of power to operate the HVAC system in the one or more zones.