Provided is an information processing system including a reflection member, a hand-held information processing apparatus including an imaging section, and an information processing apparatus communicable with the hand-held information processing apparatus. The reflection member is a retroreflecting material. The hand-held information processing apparatus includes the imaging section and a light emitting section. The hand-held information processing apparatus analyzes a captured image, calculates the position of an image of the retroreflecting material corresponding to reflected light from the retroreflecting material, and outputs positional information thereof. The information processing apparatus estimates the orientation or position of the hand-held information processing apparatus on the basis of the positional information, and generates an image based on the result of the estimation.