A novel layered orthographic representation of the light field, comprising a set of 3-D orientations, each orientation having an associated depth direction and two sampling directions, each such orientation being associated with a set of planar grids normal to the depth direction and containing grid points evenly sampled along the sampling directions, each grid containing orthographic samples of the light field intercepting that grid point in the direction of the associated depth direction. Information of the geometric structure is similarly stored in depth field format at these sample points.