A hydraulic drive system for storing and releasing hydraulic fluid includes a high pressure storage device, a low pressure storage device, and a pump-motor operating at a range of pump-motor speeds for converting between hydraulic energy and mechanical energy. The pump-motor is disposed between the high pressure device and the low pressure device. In normal operation, the hydraulic drive system enters a motoring mode where hydraulic energy is released from the high pressure storage device and converted to mechanical energy using the pump-motor. It also enters a pumping mode where mechanical energy is converted into hydraulic energy. A neutral state exists where hydraulic energy is neither stored nor released from the high pressure storage device. When the pump-motor includes a swash-plate or similar structure, compensating for drift during normal operation helps to promote efficient operation of the hydraulic drive system.