An endoscope inserting direction detecting system receives an endoscopic image, detects an endoscope inserting direction, in which an endoscope should be inserted, on the basis of the endoscopic image and produces information concerning the detected inserting direction. The endoscope inserting direction detecting method includes setting sampling pixels, detecting the direction of a change in brightness in the endoscopic image and evaluating the angles among a plurality of inserting-direction candidates and the direction of the change in brightness.