Disclosed are systems for and methods of registering (i.e., aligning) a deformable image with a reference image subject to a spatially variant flexibility model and/or a non-Gaussian smoothing model. These systems and methods allow for the consideration of differences between the ways in which structures of interest may move within a patient. The registration includes modifying the deformable image to match similar features in the reference image. The systems include a deformation engine configured for performing a deformation algorithm subject to the spatially variant flexibility and/or smoothing models.