In a robot safety system, a safety controller specifies a robot that a worker is looking at based on position of worker and/or visual field of worker, and acquires distance between robot and worker based on position information. Safety controller generates a fence image of which display mode is changed depending on distance to robot, which is a virtual image showing a boundary position of an operating range of robot, based on acquired distance. Generated fence image is displayed on a glasses type display device that the working is wearing such that fence image overlaps visual field of worker. In another example of the robot safety system, a virtual fence may be displayed on a head mount type display device as a fence image, and display mode of fence image may be optimized such that work efficiency of worker is affected as little as possible.