A method for scaling video content based on bandwidth rate is disclosed. Wherein, a plurality of target shots and non-target shots are recognized from shots of a video. A background and at least one foreground object are separated from the video frame of the target shot. The present invention method provides a plurality of bandwidth levels, and one among them is selected in advance. At least one foreground object is selected and pasted on the background based on the selected bandwidth level, so as to obtain at least one recombined target shot. The recombined target shot, the non-target shot or other recombined target shot can be merged to form a recombined video unit based on the selected bandwidth level. Therefore, the present invention proposes video content of scalability based on bandwidth rate to provide the different video bit-stream size, so that viewer can get the high quality video under the lower bandwidth rate.