The present disclosure provides an organic light emitting diode (OLED) substrate and a manufacturing method thereof, and a display device. The OLED substrate includes: a base substrate; a pixel defining layer on the base substrate, the pixel defining layer including pixel defining patterns for defining sub-pixel units, each sub-pixel unit being defined between adjacent two of the pixel defining patterns; and an organic light emitting layer on a side of the pixel defining layer away from the base substrate, the organic light emitting layer including a first portion in each sub-pixel unit and a second portion on each pixel defining pattern. Each pixel defining pattern is provided with a groove structure therein, and part of the second portion of the organic light emitting layer in the groove structure and the other part of the second portion of the organic light emitting layer are spaced apart from each other.