A fixing device includes a fixing roller, a pressure roller and a soaking roller. Temperature of the soaking roller, which contacts with the pressure roller, is measured by a temperature measurement section. This makes it possible to precisely estimate the temperature distribution of the fixing roller and the pressure roller including a non-paper feed area of paper sheets. Based on the temperature of the soaking roller, fixing operation is controlled by a fixing operation control section. Thereby, optimal control can be implemented to suppress the temperature rise in the non-paper feed area according to the estimated temperature distribution of the fixing roller and the pressure roller.