A thermostat system with a thermostat control program is disclosed for controlling a Heating Ventilation and/or Air Conditioning (HVAC) system which incorporates a mechanism for detecting activity or occupancy in a room, area or conditioned space served by the HVAC system. The thermostat control program analyzes levels, counts or other aspects of activity detected in the conditioned space, with an operating sequence which may include pattern recognition techniques. The operating sequence of the thermostat control program may further depend upon time of day, and upon periods of time identified as being periods for special handling of occupancy, or the recognition of occupancy. These factors may then be utilized by the thermostat control program to influence determination of the temperature setpoint, or to select from alternative programming provided either by the user of the thermostat, or by factory programming, with purpose of balancing energy savings and comfort.