Systems and methods of automating the generation of an estimate of the first floor elevation of a building, using a Digital Terrain Model (DTM) and an image of the front of the building. First, machine learning (ML) algorithms are used to estimate the height of the first floor of the building above ground from the image. The estimated height is added to a DTM height of the geographical area corresponding to the location of the building to provide an estimate of the FFE. The disclosed embodiments can be applied in estimating the first floor height of any kind of building. For example, single-family homes, townhomes, high rise buildings, stores, malls, shops, movie theaters, or any other building. Additionally, the disclosed embodiments are camera-agnostic and can be applied on any type of image collected by any type of camera.