A method of forming an isolation layer of a semiconductor device is disclosed herein, the method comprising the steps of providing a semiconductor substrate in which a tunnel insulating layer and a charge storage layer are formed on an active area and a trench is formed on an isolation area; forming a first insulating layer for filling a lower portion of the trench; forming a porous second insulating layer on the first insulating layer for filling a space between the charge storage layers; forming a third insulating layer on a side wall of the trench and the second insulating layer, the third insulating layer having a density higher than that of the second insulating layer; and forming a porous fourth insulating layer for filling the trench.