A sheet conveying device includes a shift unit. The shift unit includes a gate member configured to align a recording medium being conveyed thereto by causing a leading end of the recording medium to abut against the gate member so that part of the recording medium bends to form a curved portion; a pair of registration rollers configured to convey the recording medium aligned by the gate member at a predetermined timing, the pair of registration rollers including a driving roller and a driven roller, the driving roller having a rotating shaft coaxial with a rotational central shaft of the gate member; and a drive transmitting unit configured to transmit rotational drive from a drive unit to the driving roller, the drive transmitting unit being disposed on an end side of the rotating shaft of the driving roller.