Methods, systems and apparatus, including computer programs encoded on computer storage media, for video content classification. A video content classification system classifies input video data frame by frame. The video content classification system performs customized discrete cosine transform (DCT) operations on each frame. The video content classification system then determines, based on results of the DCT operations, how much image information in the frame may be lost due to compression. If the loss is estimated to be below a threshold value, the video content classification system applies a high data compression ratio algorithm to compress the frame. Otherwise, the video content classification system applies a low data compression ratio algorithm to compress the frame. The video content classification system may generate output video data that is compressed differently frame by frame.