Methods and systems are described for combining sources in a data compression system. In a system in which a context model results in the production of multiple sources or symbols, each source being associated with a probability estimate, sources may be combined by defining another, reduced size, set of probabilities. The new set of probabilities may be a subset of the predefined set of probabilities provided by the context model. Minimizing relative entropy may be a basis upon which to define a mapping of predefined probabilities to the new set of probabilities. An input sequence that was modeled using the context model may then be entropy encoded and entropy decoded using the new set of probabilities based on a mapping between the new probabilities and the predefined probabilities.