An image processing device includes a drive unit connected with an ultrasound element or an image sensor element, and an image processing unit that sequentially generates two-dimensional images of an organ, a blood vessel, or a medical instrument, based on information which is acquired by the ultrasound element or the image sensor element, and that generates a three-dimensional image based on the two-dimensional image. The image processing unit includes a storage unit that stores position correspondence information and/or image correspondence information, and a control unit that determines a position inside the three-dimensional image corresponding to a current position of the ultrasound element or the image sensor element, based on the position correspondence information or the image correspondence information, and that generates display information in real time, in which the determined position inside the three-dimensional image is identified and displayed inside the three-dimensional image.