A method and apparatus for video encoding or decoding performed by a video encoder or a video decoder incorporating advanced multiple transform (AMT) are disclosed. According to this method, a transform set consisting of two type candidates designated as a first type and a second type is determined, where the first type and the second type are different. If AMT is used by the current block, a vertical transform is selected from the transform set and a horizontal transform is selected from the transform set. A transform index, for indicating both vertical transform selection and horizontal transform selection, can be signaled at the encoder side or the transform index is parsed at the decoder side. Another method and apparatus incorporating AMT are also disclosed where the transform set is determined according to the resolution of current MVD.