In a method of forming an integrated circuit device, an opening is formed extending through a first and a second insulating layers and through a semiconductor layer therebetween to a surface of a substrate. The opening includes a recess in a sidewall thereof between the first and second insulating layers adjacent the semiconductor layer. A conductive plug is formed on the sidewall of the opening and on the surface of the substrate and laterally extending into the recess between the first and second insulating layers to contact the semiconductor layer. The semiconductor layer may be selectively etched at the sidewall without substantially etching the first and second insulating layers at the sidewall of the opening to form the recess between the first and second insulating layers. Related devices are also discussed.