A video decoder can be configured to receive a syntax element indicating whether chroma scaling matrices are signaled for the video data; in response to determining that chroma scaling matrices are signaled for the video data, determine a chroma scaling matrix for a block of video data; determine a block of chroma transform coefficients for the block of video data; dequantize a first chroma transform coefficient of the block of chroma transform coefficients using a first scaling value from the chroma scaling matrix; dequantize a second chroma transform coefficient of the block of chroma transform coefficients using a second scaling value from the chroma quantization matrix; and determine a chroma residual block for the block of video data based on the first dequantized chroma transform coefficients and the second dequantized chroma transform coefficient.