A method, a system and a computer program product for using speech/voice recognition technology to update digital video recorder (DVR) program recording patterns, based on program viewer/listener feedback. A speech controlled pattern modification (SCPM) utility utilizes a DVR recording sub-system integrated with speech processing functionality to compare control phrases with phrases uttered by a viewer. If a control phrase matches a phrase uttered by the viewer, the SCPM utility modifies the DVR recording patterns, according to a set of pre-programmed governing rules. For example, the SCPM utility may avoid modifying the recording patterns for programs within a list of “favorite” programs but may modify the recording patterns for programs excluded from the list. The SCPM utility determines priority of the uttered phrases by identifying users and retrieving a preset priority level of the identified users. The priority level is then used to control changes to the recording patterns.