This hydraulic drive system includes: first and second circuit systems; first and second hydraulic pumps; a merge valve that opens and closes a merge passage connecting the hydraulic pumps; an operation device that outputs an operation command corresponding to an amount of operation specifying an amount of actuation of first and second hydraulic actuators; and a control device that controls the merge valve according to the operation command from the operation device. The first circuit system includes: a first meter-in control valve that controls a meter-in flow rate of the working fluid that flows to the first hydraulic actuator; and a first meter-out control valve that controls a meter-out flow rate of the working fluid that is drained from the first hydraulic actuator into a tank. The control device controls an opening degree of the first meter-in control valve and an opening degree of the first meter-out control valve.