A source color grade and a destination color grade may be received by an end-to-end image mapping pipeline comprising forward and backward paths. Forward neural networks in the forward path may be applied to generate, from the source color grade, a forward reshaped color grade corresponding to the destination color grade. Backward neural networks in the backward path may be applied to generate, from the forward reshaped color grade, a backward reshaped color grade corresponding to the source color grade. An overall neural network cost function may be specified for the forward path and the backward path to generate a cost that may be used as a basis for determining operational parameters for the forward and backward neural networks.