A rotary drive device includes a drive motor and drive transmission portion, which transmits a rotational drive force of the motor to a driven rotor and includes three or more drive transmission members. A high stiffness member is attached to a low-stiffness drive transmission member, which is one, lower in stiffness, of two of the drive transmission members, to increase stiffness of the low-stiffness drive transmission member. The two form a drive transmission point corresponding to a resonant point of a lowest resonance frequency among frequencies at a plurality of resonance points in frequency response characteristics derived from rotation information. One drive transmission point, which is on either a drive-transmission input side or output side, other than the drive transmission point corresponding to the resonant point of the lowest resonance frequency, of the low-stiffness drive transmission member is configured to transmit the rotational drive force through the high stiffness member.