The invention relates to a video object cooperative segmentation method based on a track directed graph. The method comprises the following steps of (1), inputting each frame sequence Fm, t(t=1, ..., Nm) of a video set; (2), generating a motion vector field, an initial significance graph and a candidate object for the video frame Fm, t; (3), performing frontward and backward tracking on each candidate object, and performing maximal inhibition and track segmentation, thereby forming a track set; (4), constructing a directional weighted graph G=(V,E), wherein the track is a directional edge which is established between nodes in the graph according to a matching score; (5), converting the directional weighted graph G=(V,E) to a non-directional weighted graph G=(V,E'), extracting a maximal clique by means of a maximal clique extracting algorithm, calculating the weighted clique score of each maximal clique, using a track area which corresponds with the clique with highest weighted clique score as a main object area, performing a popular sequencing algorithm for generating an objective significance graph, and obtaining a final segmentation result by means of GrabCut; and (6), according to the obtained object segmentation result, updating an initial significance graph, calculating the maximal clique score and obtaining the object of other kind.