Samples of a scene are acquired in synchronization with finely interleaved varying conditions such as lighting, aperture, focal length, and so forth. A time-varying sample at each pixel for each condition is reconstructed. Time multiplexed interleaving allows for real-time, live applications, while handling motion blur naturally. Effects such as real-time video relighting of a scene is possible. Time interleaved exposures also allow segmentation of a scene, and allows for constrained special effects such as triangulation matting using a multi-color interleaved chroma key.