The invention provides a hierarchical video coding method supporting multi-source data fusion. The method comprises the following steps: step 10, obtaining multi-source information of an external sensor and a synchronous video, forming a video object in the synchronous video according to a preset analysis model, and combining the multi-source information with the corresponding video object to form object description information; step 20, obtaining the allocated synchronous video, and optimizing the image in the synchronous video according to a preset optimization model to form an optimized video; and step 40, carrying out video coding on the optimized video, coding the corresponding object description information, and embedding video code stream to form hierarchical code stream of a source video. The optimized video and the multi-source information of the external sensor are coded and are synchronously combined, so that the source video forms the hierarchical code stream containing video contents and video object information. The invention further provides corresponding hierarchical video coding architecture supporting multi-source data fusion.