A method for controlling a robot. The method includes receiving an indication of a target configuration to be reached from an initial configuration of the robot, determining a coarse-scale value map by value iteration, starting from an initial coarse-scale state and until the robot reaches the target configuration or a maximum number of fine-scale states has been reached, determining a fine-scale sub-goal from the coarse-scale value map, performing, by an actuator of the robot, fine-scale control actions to reach the determined fine-scale sub-goal and obtaining sensor data to determine the fine-scale states reached, starting from a current fine-scale state of the robot and until the robot reaches the determined fine-scale sub-goal, the robot transitions to a different coarse-scale state, or a maximum sequence length of the sequence of fine-scale states has been reached and determining the next coarse-scale state.