An engine speed control device performing: a first PID gain calculation step of calculating a target engine speed to thereby calculate a first PID gain based on an engine speed deviation between the target engine speed and an engine speed; a target rack position calculation step of correcting the first PID gain based on a cooling water temperature to thereby calculate a target rack position of a fuel injection pump; a second PID gain calculation step of calculating a second PID gain based on a rack position deviation between the target rack position and a rack position; and a rack control signal producing step of correcting the second PID gain based on a lubricating oil temperature to thereby produce a rack control signal. The engine speed control device thus controls an engine speed by controlling the rack position based on the rack control signal.