An apparatus operation device includes a line-of-sight detecting unit that detects a line of sight of a user, a neck-mounted terminal that is mounted around a neck of the user and detects a motion of the neck of the user, a determining unit that determines, based on the line of sight that has been detected and the motion of the neck that has been detected, at least one of an electronic apparatus as a target apparatus to be operated or operation details, and an apparatus control unit that controls the electronic apparatus as the target apparatus in accordance with the determination.