A method and a system for adaptively (dynamically) reducing quantization layer reduction for removing quantization artifacts in quantized video signals is provided. Adaptively reducing quantization layer reduction involves detecting if a selected pixel in a quantized image belongs to a ramp area in each of multiple pre-defined directions, dynamically selecting a quantization level for each of the pre-defined directions based on the corresponding detection results, and refining the pixel based on the selected quantization levels.