A tool box assembly includes at least one case, at least one tool holder mounted in and rotated relative to the at least one case, and multiple hand tools mounted in the at least one tool holder. The at least one case has a first pivot portion and multiple locking elements. Each of the locking elements has a second pivot portion, a connecting portion, and a press portion. The at least one tool holder has a third pivot portion. When the press portion is pressed, each of the locking elements is rotated about the connecting portion, so that the third pivot portion is detached from the second pivot portion, and the at least one tool holder is detached from the at least one case. Thus, the at least one tool holder and the hand tools are used individually.