Techniques for enhancing an image are described. For example, a lower-resolution image from a video file may be enhanced using a trained neural network applying the trained neural network on the lower-resolution image to remove artifacts by generating, using a layer of the trained neural network, a residual value based on the proper subset of the received image and at least one corresponding image portion of a previously generated higher resolution image in the video file, upscaling the received image using bilinear upsampling, and combining the upscaled received image and residual value to generate an enhanced image.