Provided are a memory device and a method of forming the same. The memory device includes a substrate, a layer stack, and a plurality of composite pillar structures. The layer stack is disposed on the substrate. The layer stack includes a plurality of conductive layers and a plurality of dielectric layers stacked alternately. The composite pillar structures respectively penetrate through the layer stack. Each composite pillar structure includes a dielectric pillar; a pair of conductive pillars penetrating through the dielectric pillar and electrically isolated from each other through a portion of the dielectric pillar; a channel layer covering both sides of the dielectric pillar and the pair of conductive pillars; a ferroelectric layer disposed between the channel layer and the layer stack; and a buffer layer disposed between the channel layer and the ferroelectric layer.