According to an embodiment, an image decoding apparatus includes a memory, a decoder and a first filter. The memory stores reference pixels based on pixels included in a decoded pixel block. The decoder decodes encoded data in units of pixel blocks using the reference pixels to generate a first decoded pixel block, the first decoded pixel block being adjacent to the reference pixels. The first filter performs a first filtering on only the first decoded pixel block using the first decoded pixel block and part of the reference pixels perpendicularly adjacent to the first decoded pixel block in a scan direction of image decoding processing.