Innovations in content mastering operations performed during playback of high dynamic range (“HDR”) video on a display device are described. When content mastering is performed during playback on a display device, a video playback system can use details retained for input HDR video (e.g., retained in metadata) and the properties of the display device to improve the perceptual quality of the HDR video as shown on that display device. For example, the video playback system can use an energy-preserving bloom operator to make bright highlights “bloom” into adjacent areas, thereby accentuating the bright highlights in the HDR video while operating within the constraints of the display device. The video playback system can also perform various other types of operations when content mastering is deferred until playback, including application of a lens flare operator as well as alternative tone mapping operators and alternative color gamut mapping operators selected according to metadata.