Tools and techniques for contact grouping and gesture recognition for surface computing are provided. These tools receive, from a finger grouping subsystem, data representing velocity parameters associated with contact points, with the contact points representing physical actions taken by users in interacting with software applications presented on a surface computing system. These tools also identify gestures corresponding to the physical actions by analyzing the velocity parameters, and provide indications of the identified gestures to the software applications.