Provided are an image processing device, an image processing method, and a program capable of forming an accurate three-dimensional map even in a case where a moving object is included in an input image. The image processing device includes an image acquiring section that sequentially acquires two-dimensional captured images, an object type recognition executing section that attaches, to each pixel in the sequentially acquired captured images, a label indicating the type of an object represented by the pixel, and a three-dimensional map creating section that executes three-dimensional position recognition of each pixel of the captured images to create a three-dimensional map, on the basis of the sequentially acquired captured images, and the three-dimensional map creating section restricts the three-dimensional position recognition of each pixel of the captured images according to the label attached to the pixel.