In one aspect, the present disclosure relates to a method including obtaining, by a fitness tracking device, a plurality of heart rate measurements of the user over a period of time, wherein the plurality of heart rate measurements can include heart rate data from a heart rate sensor of the fitness tracking device; analyzing, by the fitness tracking device, the plurality of heart rate measurements to determine a rate of change of a heart rate of the user during the period of time; determining, by the fitness tracking device, that the user is experiencing an onset phase if the rate of change of the heart rate during the period of time is greater than zero; determining, by the fitness tracking device, that the user is experiencing a cool-down phase if the rate of change of the heart rate during the period of time is less than zero; estimating, by the fitness tracking device, a first rate of energy expenditure of the user if the user is experiencing an onset phase using an onset calorimetry model; and estimating, by the fitness tracking device, a second rate of energy expenditure of the user if the user is experiencing a cool-down phase using a cool-down calorimetry model.