Methods, systems, and apparatus, including computer programs encoded on computer-storage media, for fish weight estimation based on fish tracks identified in images. In some implementations, a method includes obtaining images of fish enclosed in a fish enclosure, identifying fish tracks shown in the images of the fish, determining a quality score for each of the fish tracks, selecting a subset of the fish tracks based on the quality scores, determining a representative weight of the fish in the fish enclosure based on weights of the fish shown in the subset of the fish tracks, and outputting the representative weight for display or storage at a device connected to the one or more processors.