A displacement control device of a variable-displacement hydraulic pump which increases the response speed when the pump discharge pressure is low and suppresses increase of the response speed when the pump discharge pressure is high. The displacement control device has a variable-displacement piston 3 which activates a displacement control member 2 of the variable displacement hydraulic pump 1, a circuit 10 which provides a communication between a large-diameter pressure chamber 6 on the variable-displacement piston 3 and an output port 9 of the control valve 7, and a variable restriction valve 11 which is provided in the circuit 10. The restriction valve 11 increases and decreases its opening area when the pump discharge pressure is low and high, respectively. When the pump discharge pressure is low, the pump discharge pressure is smoothly introduced into the large-diameter pressure chamber 6 through the control valve 7, whereas, when the pump discharge pressure is high, the supply of the discharge pressure to the large-diameter pressure chamber is restricted.