The present disclosure illustrates a robot speech control system including a robot body, a handheld device, a command receiver, an ambient sensor, a processor and a controller. The handheld device includes a display interface and a speech input interface, and the speech input interface is configured to input a speech command which is then converted, by a speech recognition program, into a speech control instruction. An auto-dodge program converts the information into an auto-dodge instruction. The processor receives the speech control instruction and the information associated with the environment, and use Bayesian regression algorithm to calculate a free space probability, so as to obtain weights of a speech command and a dodge command. The processor integrates the speech control instruction and the weight of the speech command, and the auto-dodge instruction and the weight of the dodge command, to generate a movement control instruction.