The process of modeling a complex two-dimensional periodic structure is improved by selectively truncating the Fourier expansion used in the calculation of resulting scatter signature from the model. The Fourier expansion is selectively truncated by determining the contribution for each harmonic order in the Fourier transform of the permittivity function and retaining the harmonic orders with a contribution that is above a threshold. The Fourier space may be compressed so that only the selected harmonic orders are used, thereby reducing the required memory and calculation times. The compressed Fourier space may be used in a real-time analysis of a sample or to generate a library that is used in the analysis of a sample.