A combination structure of an Exhaust Gas Recirculation (EGR) cooler may include a housing open at both sides thereof, a plurality of headers inserted in both of the sides of the housing and having a circumference forming a layer with the housing, and a plurality of diffusers each including a coupling portion, which is inserted in the headers to form a layer with the housing and the headers, at a first side and having a hole for receiving and discharging exhaust gas at a second side, in which sides of the housing and the headers may be welded, and the housing, the circumference of the headers, and coupling portions of the diffusers may be welded.