An image processing device including a predicted vector generation section configured to generate a predicted vector for use in encoding of a motion vector (MV) of a current block by scaling the MV of a reference block, which is a block of a position shifted from a position of the current block in an image of a different view, by a disparity obtained from a periphery of the current block in an image of a non-base view according to a reference destination of the current block and a reference destination of the reference block, an MV encoding section configured to encode the MV of the current block using the predicted vector generated by the predicted vector generation section, and an encoding section configured to generate an encoded stream by encoding the image in units having a hierarchical structure.