According to one aspect of the present invention, a hydrogen fuel filling system includes a first flow passage through which hydrogen fuel supplied from a pressure accumulator that accumulates hydrogen fuel under pressure passes; a second flow passage through which hydrogen fuel supplied from the pressure accumulator passes, and which is arranged in parallel with the first flow passage; a switching valve that switches flow passages selectively from one of the first and second flow passages to another, or that switches flow passages between one and both of the first and second flow passages; and a control circuit that controls opening/closing of the switching valve, wherein a fuel cell vehicle using hydrogen fuel as a power source is filled with hydrogen fuel while switching the flow passages by the switching valve during supply from the pressure accumulator.