Joint coding of depth map video and texture video is provided, where a motion vector for a texture video is predicted from a respective motion vector of a depth map video or vice versa. For scalable video coding, depth map video is coded as a base layer and texture video is coded as an enhancement layer(s). Inter-layer motion prediction predicts motion in texture video from motion in depth map video. With more than one view in a bitstream (for multiview coding), depth map videos are considered monochromatic camera views and are predicted from each other. If joint multiview video model coding tools are allowed, inter-view motion skip is used to predict motion vectors of texture images from depth map images. Furthermore, scalable multiview coding is utilized, where inter-view prediction is applied between views in the same dependency layer, and inter-layer (motion) prediction is applied between layers in the same view.