The invention relates to a dual-mode brightness interpolation filter structure and method. According to the dual-mode brightness interpolation filter structure, a sub-block 1 is cascaded with two groups of sub-blocks 2 to form a first stage circuit of a dual-mode brightness interpolation filter circuit; the two groups of sub-blocks 2 are connected behind the output of the sub-block 1 and first stage circuit, and the first stage circuit is connected with a group of sub-blocks 2 to form a second stage circuit of the dual-mode brightness interpolation filter circuit; the two groups of sub-blocks 2 are connected behind second stage output and the first stage circuit, and a group of sub-blocks 2 is connected behind the output of the sub-block 1 and the second stage output to form a third stage circuit of the dual-mode brightness interpolation filter circuit; and the three stage circuits form a final dual-mode brightness interpolation filter circuit. In the dual-mode brightness interpolation filter structure and method, according to different encoding modes of preceding stage input, the H.264 protocol and the AVS/SVAC protocol are reused in the brightness interpolation filter circuit to improve the utilization rate of on chip resources and reserve domain volumes for other modules of an encoder; and moreover, by means of the cascading mode, sub-pixel values of various types are obtained by filtering in a flow form, and an inter-frame prediction brightness interpolation filter process in video coding can be accomplished quickly and efficiently.