An eye tracking system for a NED device includes sensors that are directed toward and angularly offset from a user's eyes in a manner that causes circular features (e.g., irises and/or pupils) of the user's eyes to appear elliptical within sensor planes that correspond to the individual sensors. The eye tracking system determines parameters associated with detected ellipses and then uses these parameters to generate 3D propagations from the detected ellipses back to the user's eyes. By analyzing these 3D propagations with respect to ocular rotation models that represent how the pupils and iris rotate about the center of the eye, the eye tracking system determines pupil orientation parameters that define physical characteristics of the user's eyes. The pupil orientation parameters may then be used to determine interpupillary distance and/or vergence of the user's visual axis that extend from the user's fovea.