System, methods, and other embodiments described herein relate to tracking dynamic objects in a surrounding environment of a vehicle. In one embodiment, a method includes, in response to acquiring sensor data from at least one sensor, generating a current occupancy map that indicates locations of occupied grid cells as identified by the sensor data. The method includes updating a difference map according to the current occupancy map. The difference map encodes temporal changes in relation to prior states of occupancy of the grid cells to track dynamic objects in the surrounding environment over a defined temporal horizon. The method includes computing dynamics of the dynamic objects according to the difference map. The method includes providing the dynamics to at least one vehicle system within the vehicle.