Methods and apparatus of motion compensation using the bi-directional optical flow (BIO) techniques are disclosed. According to one method of the present invention, the BIO process is applied to encode or decode bi-directional current block in Merge mode only or in AMVP (advanced motion vector prediction) mode only. According to another method, the BIO process conditionally to encode or decode the current block depending on a jointly-coded flag if the current block is coded using a bi-prediction mode. According to yet another method, x-offset value vx and y-offset value vy for the current block are added to the current motion vector to form a final motion vector. The final motion vector is then used as a reference motion vector for following blocks. In still yet another method, the BIO process is applied to the chroma component.