A filter processing section 352 performs a filter process for applying an offset to pixels of a decoded image. A filter controller 351 inhibits the offset from being applied to an edge portion depending on occurrence of a transform unit in a transformation skip mode in which orthogonal transformation is not performed on a predicted residual. Furthermore, the filter controller 351 determines whether an offset for gradation adjustment using the decoded image is to be applied in the filter processing section 352 or not, and sets an offset in case the offset is to be applied. It is possible to make less conspicuous in the boundary between a transform unit where orthogonal transformation is performed by the offset for gradation adjustment and the transform unit where orthogonal transformation is skipped. The image quality of the decoded image is restrained from being lowered.