Embodiments are directed to verifying media stream quality for multiparty video conferences. A verification video may be generated based on verification goals for a video provided by a video service. A marker may be embedded in the verification video. A video conference may be established using video stations such that the video conference may be provided by a video service. The verification video may be streamed to a video input of each video station. The video may be streamed to a video output buffer of each video station such that the video provides a view of the video conference and such that the marker that corresponds to each video station may be included in the video. Video information may be captured from the video output buffer of the video stations. The video service may be classified based on the video information from each video station.