A device for coding video data includes a memory storing video data and a video coder including one or more processors configured to determine a current coding unit of the video data is coded in a palette mode; determine a palette for the coding unit by, for a first entry of the palette, choosing a predictor sample from a reconstructed neighboring block of the coding unit and coding a difference between one or more color values of the first entry and one or more color values of the predictor sample.