A method of filtering an input sample stream having a downsampling rate is disclosed to generate an output sample stream having an upsampling rate that is less than the downsampling rate. The input sample stream is input to a rate change filter having multiple filter branches. The input sample stream is filtered at each of the multiple filter branches to output filtered sample substreams. Each of the multiple filter branches have filter coefficients corresponding to a different phase of the filter response. The filtered sample substreams are stored in a memory and the stored filtered sample substreams are combined to generate the output sample stream.