A sheet conveying device includes a first conveying path configured to pass a sheet of a recording medium (e.g., paper) therethrough to a sheet processing device, a second conveying path branched from the first conveying path and configured to temporarily store the paper sheet conveyed therein, a sheet conveying mechanism configured to selectably convey the paper sheet in one of forward and backward directions to the sheet processing device, a guide member mounted at a branch point of the first and second conveying paths and configured to guide the paper sheet when the paper sheet is conveyed in the backward direction by the sheet conveying mechanism to the second conveying path, and a control unit configured to control the sheet conveying mechanism to change a distance between the branch point and the sheet conveying mechanism according to a length of the paper sheet in a forward sheet conveying direction.