An electrical submersible pump assembly has a pump driven by a motor. A pressure compensator has first and second bellows units axially separated from each other. Each of the first and second bellows units are movable between an increased volume position and a decreased volume position and have a bias toward the decreased volume position. The bias of the first bellows unit is greater than the bias of the second bellows unit. The greater bias of the first bellows unit over the second bellows unit causes the second bellows unit to be at a full volume position at a lower level of the pressure differential than the level at which the first bellows unit is at the full volume position.