Using scan conversion processing of changing the scan order for each block, parallel scan conversion processing is executed if possible. A scan status holding unit holds statistical information based on the appearance frequency values of coefficients in a block. A scan order holding unit holds coefficient position information in which the coefficient positions in a block are arranged based on the scan order. A significant data position information generation unit scans one block data in accordance with coefficient position information, and generates information representing nonzero/zero for each data position. Based on the statistical information and the information generated by the significant data position information generation unit, a parallel number determination unit determines whether to process two blocks in parallel or process one block. In accordance with the determination, a scan conversion unit scan-converts two input blocks in parallel or scan-converts only one of the blocks.