A computing device determines whether a prediction unit (PU) in a B slice is restricted to uni-directional inter prediction. In addition, the computing device generates a merge candidate list for the PU and determines a selected merge candidate in the merge candidate list. If the PU is restricted to uni-directional inter prediction, the computing device generates a predictive video block for the PU based on no more than one reference block associated with motion information specified by the selected merge candidate. If the PU is not restricted to uni-directional inter prediction, the computing device generates the predictive video block for the PU based on one or more reference blocks associated with the motion information specified by the selected merge candidate.