Improved 4D imaging reconstruction is provided for a freely breathing patient. 3D patient images from an imaging dataset are binned according to respiratory displacement or phase. The bins are defined by ranges, so every image in the raw 3D data set is included in a bin. Since binning in this manner often results in two or more images per bin, the 4D reconstruction is determined by selecting one 3D image from each bin at each patient position. This selection is performed so as to maximize the anatomical similarity of 3D images at adjacent patient positions. In cases where the 3D images include multiple slices, a 2D comparison of the closest slices can be used to determine anatomical similarity of the 3D images.