A data processing apparatus provides predictions for hard to predict instructions. Prediction circuitry generates predictions relating to predictable instructions in a stream and stores, with respect to each of the predictable instructions, a reference to a set of monitored instructions in the stream to be used for generating predictions for the predictable instructions. Processing circuitry executes the predictable instructions in the stream using the predictions. A given correlation parameter is stored between a given predictable instruction in the stream and a subset of the set of monitored instructions of the given predictable instruction to assist in generating the predictions. If the given correlation parameter is currently stored, the prediction circuitry generates a given prediction relating to the given predictable instruction based on the subset of the set of monitored instructions. Otherwise, the prediction circuitry generates the given prediction relating to the given predictable instruction based on the set of monitored instructions.