A graphics processing system for a head mounted display (or other non-standard projection display) comprises a low latency distortion unit which is separate from a graphics processing unit in the graphics processing system. The low latency distortion unit receives pixel data generated by the graphics processing system using a standard projection and performs a mapping operation to introduce distortion which is dependent upon the optical properties of the optical arrangement within the head mounted display. The distorted pixel data which is generated by the low latency distortion unit is then output to the display in the head mounted display.