In one embodiment, a computing system may access color components of a pixel region in an image, and then determine a color variance for each of the color components. The computing system may further determine a desired bit allocation for each of the color components based on the color variance associated with that color component. The computing system may then determine a total bit allocation for the pixel region based on the desired bit allocations for the color components, as well as a number of unallocated bits available for allocation. The computing system may further determine a final bit allocation for each of the color components by allocating the total bit allocation to each of the color components according to the desired bit allocation for each of the color components. The computing system may then encode each of the color components using the associated final bit allocation.