Techniques are described for implementing automated control systems to control operations of target physical systems and/or their components (e.g., a fuel cell, wind turbine, HVAC unit, etc.), such as based at least in part on models of their dynamic non-linear behaviors that are generated by gathering and analyzing information about their operations under varying conditions. The techniques may include, for each of multiple levels of inputs to the system/component and/or other factors, injecting a corresponding signal input into the system/component, and using active sensors to collect time changes of the responses to these pulses. Information about the inputs and the responses is used to generate an incremental parametric model representing the internal state and behavioral dynamics of the system/component, which is further used to control additional ongoing operations of the system/component (e.g., to control whether and how much output is produced in a current or future time period).