A free-form lens (for example a phase modulator, lens or deformable mirror) may be made to reproduce a light pattern specified by image data. Source regions on the free-form lens are mapped to target regions areas on an image. Areas of the source regions are adjusted to vary the amount of light delivered to each of the target regions. Adjustment of the source areas may be achieved using a L-BFGS optimization which preferably incorporates smoothness and curl regularizers. Embodiments apply parallel processing to obtain control values for a free form lens in real time or near real time. Apparatus may process image data and display an image by controlling a dynamically variable free form lens using the processed image data.