In various implementations, upon receiving a given voice command from a user, a voice-based trigger may be selected from a library of voice-based triggers previously used across a population of users. The library may include association(s) between each voice-based trigger and responsive action(s) previously performed in response to the voice-based trigger. The selecting may be based on a measure of similarity between the given voice command and the selected voice-based trigger. One or more responsive actions associated with the selected voice-based trigger in the library may be determined. Based on the one or more responsive actions, current responsive action(s) may be performed by the client device. Feedback associated with performance of the current responsive action(s) may be received from the user and used to alter a strength of an association between the selected voice-based trigger and the one or more responsive actions, wherein the altering includes incrementing or decrementing a count corresponding to the strength of association based on the feedback being positive or negative respectively.