A point group data processing device includes: an image data acquisition unit configured to acquire a captured image; a point group data acquisition unit configured to acquire point group data indicating position information of a point group corresponding to a plurality of points included in the image; an area setting unit configured to set a target area which is an area surrounding a subject on the image and an enlargement area which is an area obtained by enlarging the target area; and a target point group specifying unit configured to specify a target point group corresponding to the subject based on depth information of a point group included in the target area and depth information of a point group included in the enlargement area, which are included in the point group data.