A first sorting unit includes a second sorting unit that sorts first frequency data for luminance based on a first table, a third sorting unit that sorts second frequency data for chrominance based on a second table, a fourth sorting unit that sorts third frequency data for chrominance based on a third table, and an updating unit that updates the second and third tables based on nonzero information on the first and second frequency data before the third and fourth sorting units start sorting.