An apparatus and method are provided for controlling branch prediction. The apparatus has processing circuitry for executing instructions, and branch prediction circuitry that comprises a plurality of branch prediction mechanisms used to predict target addresses for branch instructions to be executed by the processing circuitry. The branch instructions comprise a plurality of branch types, where one branch type is a return instruction. The branch prediction mechanisms include a return prediction mechanism used by default to predict a target address when a return instruction is detected by the branch prediction circuitry. However, the branch prediction circuitry is responsive to a trigger condition indicative of misprediction of the target address when using the return prediction mechanism to predict the target address for a given return instruction, to switch to using an alternative branch prediction mechanism for predicting the target address for the given return instruction. This has been found to improve performance in certain situations.