One embodiment of the present invention provides a system that performs a carry-save division operation that divides a numerator, N, by a denominator, D, to produce an approximation of the quotient, Q=N/D. The system approximates Q by iteratively selecting an operation to perform based on higher order bits of a remainder, r, and then performing the operation, wherein the operation can include, subtracting D from r and adding a coefficient c to a quotient calculated thus far q, or adding D to r and subtracting c from q. These subtraction and addition operations maintain r and q in carry-save form, which eliminates the need for carry propagation and thereby speeds up the division operation. Furthermore, the selection logic is simpler than previous SRT division implementations, which provides another important speed up.