An endoscope inserting direction method includes a first step of receiving endoscopic images time-sequentially, a second step of sampling pixels representing low densities from pixels of each of the endoscopic images which are time-sequentially received in the first step and a third step of determining whether or not number of pixels representing low densities which are sampled in the second step is equal to or larger than predetermined number of pixels. In a fourth step, if the number of pixels representing low densities is determined to be equal to or larger than the predetermined number of pixels in the third step, a position of the barycenter of the pixels representing low densities is obtained, and based on a change in the position of the barycenter of the pixels representing low densities of the time-sequentially received endoscopic images, a direction of a shift in the time-sequentially received plurality of endoscopic images is detected. In a fifth step, an endoscope inserting direction, in which an endoscope should be inserted, on the basis of the result of the detection performed in the fourth step is determined.