A method of encoding image data performed by at least one processor, may include: receiving information regarding a data block of an image; performing control so that a residual block composed of residual signals after intra or inter prediction is generated; identifying one or more hybrid transform kernel that are available for encoding the residual block based on associations between prediction modes and sizes of residual blocks; selecting a hybrid transform kernel from among the one or more hybrid transform kernel that are available for encoding the residual block; and performing transform coding of the residual block using the selected hybrid transform kernel.