The present invention relates to the technical field of image processing, and in particular, to a patch net model for image representation and a construction method of the patch net model. The patch net model for image representation is of a forest-shaped structure consisting of a plurality of composite nodes and basic nodes, each composite node is a non-leaf node, and each basic node is a leaf node; the basic node includes a certain patch region of an image and a representative patch representing an apparent feature of the patch region; the composite node includes a certain patch region of the image and can be further decomposed into basic nodes and/or composite nodes; an edge exists between two nodes, which are located on the same layer of the forest-shaped structure and are spatially connected, and a relation matrix used for expressing the spatial relative position of the two nodes is arranged on the edge.