A method controls a robot. One or more processors receive sensor readings from one or more sensors that are monitoring a human in real time, where the human is currently observing a robotic action by a robot, and where the robotic action is a physical movement performed by the robot. The processor(s) determine a cognitive state of the human while the human is observing the robotic action by the robot, and then adjust the robotic action being performed by the robot based on the cognitive state of the human.