A method and system for compressing videos using deep learning is disclosed. The method includes segmenting each of a plurality of frames associated with a video into a plurality of super blocks. The method further includes determining a block size for partition of each of the plurality of super blocks into a plurality of sub blocks, based on a feature of each of the plurality of super blocks using a Convolutional Neural Network (CNN). The method further includes generating a prediction data for each of the plurality of sub blocks based on a motion vector predicted and learned by the CNN. The method further includes determining a residual data for each of the plurality of sub blocks by subtracting the prediction data from an associated original data. The method includes generating a transformed quantized residual data using each of a transformation algorithm and a quantization algorithm.