A ceramic matrix composite (CMC) structure (50) with first (26) and second (28) CMC walls joined at an intersection (34) containing continuous fibers (53). A gusset (52) is formed in the intersection by an inward bending of some or all ceramic fibers (53) of the intersection, resulting in a diagonal brace between the first and second CMC walls. This creates a depression (54) or void (59) in the intersection. One or more ceramic reinforcement devices fill or span the depression to prevent distortion of the gusset. The reinforcement devices may include a ceramic filler (60) or core (61), a CMC rod or cord (56), and a CMC tape (62). The ceramic filler (60) may be continuous with a ceramic insulation layer (36) on an outer surface of the first CMC wall.