Methods, systems, and media for image processing using hierarchical expansion are provided. More particularly, the hierarchical expansion can include a merge expansion and/or a refine expansion. In some embodiments, a method for image processing, comprising: receiving a plurality of images, wherein each of the plurality of images has one of a plurality of resolutions; for a first image from the plurality of images that has a first resolution, determining a first set of labels corresponding to each pixel of the first image; and for a second image from the plurality of images that has a second resolution, generating a second set of labels corresponding to each pixel of the second image based on the first set of labels.