Provided are a computer program product, system, and method for determining a skew error signal (SES) offset used to determine an SES to adjust heads in a drive unit. A determination is made of a first difference in a first orientation with respect to a direction of movement of the recordable storage media based on first and second position information read by first and second servo read elements on a first head. A determination is made of a second difference in a second orientation with respect to the direction of movement of the recordable storage media based on third and fourth position information read by the first servo read element and a third servo read element on a second head. An offset, calculated based on the determined first and second difference, is used to generate an error signal to adjust the first and second heads.