Described herein are various technologies that pertain to controlling an AV based upon forward-looking observations of road surface behavior. With more specificity, technologies described herein pertain to controlling maneuvering of an AV in a region of a driving environment based upon an observed acceleration of an object in that region of the driving environment. A plurality of positions of an object in the driving environment are determined from output of sensors mounted on the AV. An acceleration of the object is computed based upon the positions. The AV is subsequently controlled based upon the computed acceleration.