A display substrate and a manufacturing method thereof, and a display device. The display substrate includes: a base substrate including a display region and a peripheral region surrounding the display region; a first light emitting element in the display region of the base substrate; and a color film structure on a display side of the first light emitting element. The color film structure includes: a first color film, including only a first pixel color film located in the display region; and a second color film, including a second pixel color film located in the display region and at least partially non-overlapping with the first pixel color film, and a first frame color film located in the peripheral region and surrounding the display region.