A downsampler 101 converts input data having a sampling rate 2*FH to a sampling rate 2*FL which is lower than the sampling rate 2*FH. A base layer coder 102 encodes the input data having the sampling rate 2*FL in predetermined base frame units. A local decoder 103 decodes a first coded code. An upsampler 104 increases the sampling rate of the decoded signal to 2*FH. A subtractor 106 subtracts the decoded signal from the input signal and regards the subtraction result as a residual signal. A frame divider 107 divides the residual signal into enhancement frames having a shorter time length than that of the base frame. An enhancement layer coder 108 encodes the residual signal divided into the enhancement frames and outputs a second coded code obtained by this coding to a multiplexer 109.