A parallax calculating apparatus is provided with: a processor configured to perform, on each of a plurality of types of block shapes that are different from each other, (i) a parallax calculation process, (ii) a block group setting process, and (iii) a cost calculation process, by using a pair of images photographed by a stereo camera; and a selector configured to perform, on each of a plurality of pixels that constitute the image, a parallax selection process of setting a parallax value of a block in which a cost associated by the cost calculation process is minimum, as a parallax of a target pixel in which the parallax is to be obtained, from among a plurality of blocks, each of which includes the target pixel and which respectively correspond to the plurality of types of block shapes.