A jig structure for assembling a high frequency connector includes a base on which a rotatable seat is provided, and the surface area of the rotatable seat is smaller than the base. A fixed table is provided on the rotatable seat. The fixed table further includes a first table, and an adjusting rod and a plurality of guide rods are provided on the first table. A plurality of assembly tables are provided on the adjusting rod and the guide rods. The assembly tables are correspondingly disposed, and their movement is controlled by the adjusting rod. A first assembly table and a second assembly table are provided on both sides of the fixed table. The first assembly table has movements along X-axis and Y-axis, and the second assembly stage has movements along Y-axis, thereby corresponding to the relative position of the assembly table and the fixed table so as to quickly place the circuit boards and connectors to be assemble.