Systems and methods are disclosed to receive an image depicting at least a part of a document and identify a plurality of partition points dividing the image into potential segments; generate a linear partition graph (LPG) comprising a plurality of vertices using the plurality of partition points and a plurality of arcs connecting the plurality of vertices; identify a path of the LPG having a value of a quality metric above a threshold value, wherein the path is selected from a plurality of paths of the LPG and comprises one or more arcs and the value of the quality metric is derived using a neural network classifying each of a plurality of pixels of the image; and generate one or more blocks of the image wherein each of the one or more blocks corresponds to an arc of the identified path and represents a portion of the image associated with a type of an object.