Systems and techniques are described for optimizing a circuit design by using a numerical solver. The gates sizes are optimized by modeling a set of gate optimization problems and solving the set of gate optimization problems by using a numerical solver. Modeling each gate optimization problem can include selecting a portion of the circuit design that includes a driver gate that drives one or more inputs of each gate in a set of gates, and modeling a gate optimization problem for the portion of the circuit design based on circuit information for the portion of the circuit design. A differentiable objective function for delay can be created using numerical models for the delays in the circuit. In some embodiments, gradients of the differentiable objective function can be computed to enable the use of a conjugate-gradient-based numerical solver.