According to an embodiment, a transformation matrix deriving device includes a first trajectory generator, a second trajectory generator, a trajectory matcher, and a deriver. The first trajectory generator is configured to generate a first trajectory of a moving object, detected from a video, in a first coordinate system. The second trajectory generator is configured to generate a second trajectory of a moving object in a second coordinate system, from time-series data of positional information. The trajectory matcher is configured to associate the first trajectory with the second trajectory to be a motion trajectory of the same moving object based on a similarity between the first trajectory and the second trajectory. The deriver is configured to derive a transformation matrix that transforms the second coordinate system into the first coordinate system by using the first trajectory and the second trajectory associated with each other.