A method of decoding a video signal is disclosed. The present invention includes determining whether to store a first partial picture when the first partial picture and a first full picture are corresponding to a first temporal point and storing the first partial picture for decoding a second full picture referring to the first partial picture, the second full picture being corresponding to a second temporal point, the second temporal point being located after the first temporal point, wherein a level of the first partial picture on a scalable domain is lower than a level of the second full picture on the scalable domain.