An eye tracking system, images the surface (e.g., sclera) of each eye of a user to capture an optical flow field resulting from a texture of the imaged surface. The eye tracking system includes illumination source (e.g., laser) and a detector (e.g., camera). The source illuminates a portion of the eye that is imaged the camera. As the eye moves, different areas of the eye are imaged, allowing generation of a map of a portion of the eye. An image of a portion of the eye is includes a diffraction pattern (i.e., the optical flow) corresponding to the portion of the eye. Through a calibration process, the optical flow is mapped to a location where the eye is looking.