A vehicle operating device includes a portable display unit configured to display a state of a vehicle, a portable input unit configured to receive operation information of the vehicle input thereto on the basis of a displayed content by the display unit, a first information processing unit configured to process first information among input operation information, a second information processing unit configured to process second information, which is different from the first information, among the input operation information, an interlock system configured to enable only the first information to be processed in an operating state and enable the first information and the second information to be processed in a released state, and a portable wireless output unit configured to output an operation signal of the vehicle based only on the first information processed in the operating state.