Systems and methods for provisioning real-time three-dimensional maps, including: at a first processing stage: updating a three-dimensional map of an environment based on received point cloud data; extracting three-dimensional proposals of objects of the environment; projecting the three-dimensional proposals onto a two-dimensional image of the environment; generating two-dimensional proposals of the objects of the environment; at a second processing stage: detecting the objects of the environment from the two-dimensional image of the environment; generating two-dimensional bounding boxes of the objects of the environment; matching the two-dimensional bounding box with a particular two-dimensional proposal of the two-dimensional proposals; and labeling, for each two-dimensional proposal that is matched to a two-dimensional bounding box, the three-dimensional proposal corresponding to the two-dimensional proposal with semantic information associated with the two-dimensional bounding box that is matched to the two-dimensional proposal to update the three-dimensional map.