Systems and methods for determining flows by acquiring a video of the flows with a camera, wherein the flows are pedestrians in a scene. The video includes a set of frames, wherein motion vectors are extracted from each frame in the set, and a data matrix is constructed from the motion vectors in the set of frames. A low rank Koopman operator can be determined from the data matrix and a spectrum of the low rank Koopman operator can be analyzed to determine a set of Koopman modes. Then, the frames are segmented into independent flows according to a clustering of the Koopman modes.