There is provided mechanisms for scheduling communication with terminal devices. A method is performed by a network node. The method comprises receiving, in response to having performed a beam management procedure involving transmission of reference signals in a set of beams, a beam report from each of the terminal devices. Each beam report indicates one of the beams as top ranked. The method comprises scheduling communication for the terminal devices in beams of the set of beams. At least a first terminal device of the terminal devices and a second terminal device of the terminal devices are both scheduled in a first beam of the beams not indicated as top ranked in the beam report of the second terminal device.