Provided are a shape inspection device, a processing device, a height image processing method, and a height image processing program capable of accurately inspecting a measurement object. A profile data generation unit sequentially generates a plurality of pieces of profile data as the measurement object relatively moves in a Y-axis direction. A height image generation unit extracts characteristic points for the respective pieces of profile data, and moves the respective pieces of profile data in a plane intersecting with a Y axis such that the extracted characteristic points are aligned in a line in a direction corresponding to the Y axis. Then, the height image generation unit arranges the moved profile data in a direction corresponding to the Y axis to correct a height image.