Provided are a head-mounted information processing apparatus capable of improving user's convenience, and its controlling method. Therefore, it includes a main body device 2 used in a state of being worn on a user's head, and a controller used in a state of being worn on a user's wrist or hand and controlling the main body device. For example, a directional detector 91 in the main body device 2 detects a controller's direction by calculation based on a detected result of a position/attitude sensor of the controller. A rotational detector 92 detects controller's rotation, which uses the direction of the controller as a rotational axis, by calculation based on the detected result of the position/attitude sensor of the controller. An operation instruction converter 87 converts detected results of the directional detector 91 and the rotational detector 92 into instructions for the main body device 2.