Disclosed herein is a segmental ultrasonic cleaning apparatus configured to remove scales and/or sludge deposited on a tube sheet. The segmental ultrasonic cleaning apparatus includes a plurality of segment groups arranged in a ring shape on a top surface of a tube sheet along an inner wall of the steam generator, in which each of the segment groups includes an ultrasonic element segment and a guide rail support segment loosely connected to each other by metal wires through hand holes located at a lower portion of the steam generator, such that ultrasound waves radiated from an ultrasonic transducer in each of the ultrasonic element segments travels along the surface of the tube sheet, with the segment groups tightly connected in the ring shape by tightening the metal wires via wire pulleys of flange units.