An image processing device is provided. The image processing device includes a decoding unit that decodes scaling list data to generate a first quantizing matrix of a first size, a generating unit that generates a second quantizing matrix for a transform block of a second size to which zeroing of a high-frequency component is applied, by referring to only a partial matrix of the first quantizing matrix generated by the decoding unit, and an inverse quantizing unit that inversely quantizes a quantized transform coefficient of the transform block of the second size, using the second quantizing matrix generated by the generating unit.