It includes: a part which changes a computer-aided design model according to a design variable and outputs it; a part which calculates and outputs natural vibration modes and natural frequencies; a part which makes a natural vibration mode image; a part which binarizes a natural vibration mode image by a threshold value to make a binary image; a part which extracts pattern information from a binary image and outputs it together with the image; a part for distinguishing natural vibration mode which, using pattern information, distinguishes the natural vibration mode to be tracked from natural vibration modes after deformation and outputs it; and a part for calculating optimum solution which finds a design variable which makes the natural frequency of a natural vibration mode a target value and outputs it. Thereby, the structural optimization system can easily distinguish and track natural vibration modes through computer-aided design model change in structural design.