The generally Lambertian LED die emission is modified by an optical structure formed over the LED die that reduces the light emission along a normal axis and enhances the light emission at an angle with respect to the normal axis, producing a certain light emission profile. A phosphor layer is provided overlying the optical structure, such as in a reflective light-mixing chamber. The light emission profile results in more uniform flux from the LED die impinging across the surface of the phosphor. Some of the LED light passes through the phosphor and some of the light is converted to light of a different wavelength. The light emission profile results in more uniform color and temperature across the phosphor. The light emission profile may be created by a lens, a patterned layer over the LED die, a semi-reflective layer over the LED die, an optical sheet, or other technique.