System and method of timing recovery for recovering a clock signal by using adaptive channel response estimation. The channel response estimation in the timing recovery loop is dynamically adapted to the current channel response that varies over time. More particularly, the channel estimation coefficients used in a channel estimator can be adapted based on an error signal representing the difference between a received signal at the timing recovery loop and an estimated signal output from a channel estimator. Further, to prevent undesirable interaction between the channel estimator and the overall timing recovery loop with respect to clock phase recovery, the adaptation of channel estimation can be controlled in terms of speed or time so as to reduce or eliminate the channel estimator's effect on clock phase correction.