Branch prediction techniques are described that can improve the performance of pipelined microprocessors. A microprocessor with a hierarchical branch prediction structure is presented. The hierarchy of branch predictors includes: a multi-cycle predictor that provides very accurate branch predictions, but with a latency of multiple cycles; a small and simple branch predictor that can provide branch predictions for a sub-set of instructions with zero-cycle latency; and a fast, intermediate level branch predictor that provides relatively accurate branch prediction, while still having a low, but non-zero instruction prediction latency of only one cycle, for example. To improve operation, the higher accuracy, higher latency branch direction predictor and the fast, lower latency branch direction predictor can share a common target predictor.