There is provided an engine restart control device. A controller is configured to stop an engine which is adapted to generate a driving force for running a vehicle when a predetermined stop condition is met and configured to restart the engine when a predetermined restart condition is met after stopping the engine. A steering torque detector is configured to detect a steering torque of a steering wheel which is adapted to be operated by a driver. The controller includes an origin torque setting unit configured to set the steering torque detected by the steering torque detector when the predetermined stop condition is met and the engine is stopped as an origin torque. The controller restarts the engine when the steering torque detected by the steering torque detector exceeds a predetermined threshold with reference to the origin torque set by the origin torque setting unit.