An assembling structure of heat dissipation device is applied to a circuit board. A heat generation unit is disposed on one side of the circuit board. The assembling structure of the heat dissipation device includes a heat dissipation unit, at least one latch member and at least one retainer member. The heat dissipation unit is attached to one side of the heat generation unit, which side is distal from the circuit board. At least one latch section outward extends from an edge of the heat dissipation unit. The latch member is fixedly disposed on the circuit board and formed with at least one opening and at least one perforation. The latch section is correspondingly latched in the opening. An elastic member is fitted on the retainer member. The retainer member correspondingly passes through the perforation to fix the latch member on the circuit board.