A method and apparatus or decoding prediction residues in a video coding system are disclosed. According to this method, a first syntax element to a fourth syntax element are signalled or parsed in a first coding pass for at least sub-block. The current residual block is encoded or decoded using information comprising the first syntax element, the second syntax element, the third syntax element, and the fourth syntax element. The first syntax element indicates whether an absolute value of a current transform coefficient level is greater than 0. The second syntax element indicates whether the absolute value of the current transform coefficient level is greater than 1. The third syntax element indicates a parity of the current transform coefficient level. The fourth syntax element indicates whether the absolute value of the current transform coefficient level downward shifted by 1 is greater than 1.