In an example, a video coder may determine a first layer component of a first layer of video data, wherein the first layer of video data is associated with a layer identifier. The video coder may generate at least one filtered layer component by filtering the first layer component, and assign the layer identifier of the first layer and a filtered layer component index to the at least one filtered layer component, where the filtered layer component index is different than a layer component index of the first layer component. The video coder may also add the at least one filtered layer component to a reference picture set for performing inter-layer prediction of a layer other than the first layer of video data.