A video decoder can be configured to receive, for a first chroma component of a block of the video data, information for first residual samples that correspond to a difference between a first chroma block of the first chroma component and a first prediction block of the first chroma component; determine intermediate reconstructed samples based on the first residual samples; receive, for a second chroma component of the block of video data, information for second residual samples that correspond to a difference between a second chroma block of the second chroma component and the intermediate reconstructed samples; reconstruct the first chroma block based on the first residual samples and the first prediction block; reconstruct the second chroma block based on the second residual samples and the intermediate reconstructed samples; and output decoded video data comprising the reconstructed first chroma block and the reconstructed second chroma block.