A vehicle remote instruction system 100 transmits a remote instruction request from an autonomous driving vehicle 2 to a remote instruction apparatus 1, and controls travel of the autonomous driving vehicle 2 based on a remote instruction transmitted from the remote instruction apparatus 1 in response to the remote instruction request. The vehicle remote instruction system 100 includes a delay determination unit 39 configured to determine whether or not a communication delay occurs between the remote instruction apparatus 1 and the autonomous driving vehicle 2, and a rejection unit 40 configured to reject the remote instruction transmitted in response to the remote instruction request if it is determined by the delay determination unit 39 that the communication delay occurs.