A metal is formed into an opening that is located in an interlayer dielectric (ILD) material that laterally surrounds a semiconductor fin of a partially fabricated vertical transistor and on a physically exposed topmost surface of the semiconductor fin. A patterned material stack of, and from bottom to top, a membrane and a doped amorphous semiconductor material layer is formed on the metal and a topmost surface of the ILD material. A metal induced layer exchange anneal is then employed in which the metal and doped semiconductor material change places such that the doped semiconductor material is in direct contact with the topmost surface of the semiconductor fin. The exchanged doped semiconductor material, which provides a top source/drain structure of the vertical transistor, may have a different crystalline orientation than the topmost surface of the semiconductor fin.