A rotary operation device that can prevent misassembly of a rotary operation member and a rotational force transmission member, has a rotary operation member, a rotation detector, and a rotational force transmission member. The rotary operation member has a first gear and the rotational force transmission member has a second gear and a transmission axis. One of the rotary operation member and the rotational force transmission member has a first interference portion that interferes with the other member in a position misaligned from a proper engagement position where the first gear and the second gear are engaged in a proper phase relationship. The other member comprises a second interference member that interferes with the first interference portion in a position misaligned from the proper engagement position and an escape portion that receives the first interference portion in the proper engagement position to allow assembly with the one member.