Various technologies described herein pertain to generating an output video loop from an input video that includes values at pixels over a time range. Respective input time intervals within the time range of the input video are determined for the pixels by performing an optimization. The optimization can be performed to assign the input time intervals at a first level of resolution, while terms of an objective function use a finer, second level of resolution. An input time interval for a particular pixel includes a per-pixel loop period of a single, contiguous loop at the particular pixel within the time range from the input video. The input time intervals can be temporally scaled based on per-pixel loop periods and an output video loop period. The output video loop is created based on the values at the pixels over the input time intervals for the pixels in the input video.