A mutual optimization method for class matrix and diffusion weighting used in a halftone image processing technique and a mutual optimization system thereof are provided. In the method, a mutual optimization of a plurality of diffusion weightings and a class matrix used in a dot diffusion process is performed based on a concept of simulated annealing in order to avoid converging to local solution, so as to ensure an image quality of a halftone image generated by the dot diffusion process. Besides, since the mutual optimization method is for a hexagonal class matrix, a block effect appearing in the halftone image can be significantly reduced.