A lossless compression method and system applied to hardware video decoding are provided. The method sequentially includes a video decoding step, a compression step, a storage step and a decompression step, wherein in the compression step, lossless compression is performed by taking a 16×4 Y luma block and a corresponding 8×2 U chroma block and V chroma block as a compression unit; in the storage step, compact compression and storage is performed on a complete frame of image by taking four pixel lines as a unit, and a starting address of every four pixel lines is fixed; and the video decoding step includes a reference frame reading step, and a two-level Cache structure is adopted in the reference frame reading step. The system is a system applying the method.