A replacement gate structure (i.e., functional gate structure) is formed and recessed to provide a capacitor cavity located above the recessed functional gate structure. A ferroelectric capacitor is formed in the capacitor cavity and includes a bottom electrode structure, a U-shaped ferroelectric material liner and a top electrode structure. The bottom electrode structure has a topmost surface that does not extend above the U-shaped ferroelectric material liner. A contact structure is formed above and in contact with the U-shaped ferroelectric material liner and the top electrode structure of the ferroelectric capacitor