Provided is an image processing device including a control section configured to set based on a first quantization parameter offset set for a chroma component of a first layer, a second quantization parameter offset for a chroma component of a second layer decoded with reference to the first layer, and an inverse quantization section configured to inversely quantize transform coefficient data of the chroma component of the second layer using a quantization parameter computed using the second quantization parameter offset set by the control section.