Skip to main content

Section 29.1 Motivating examples for Nonlinear Least Squares

Suppose we observe population growth over time, and expect that it should be governed by a differential equation of the form

\begin{equation} y' = ry(1-y/M)\label{eq-logistic-ode-growth}\tag{29.1.1} \end{equation}

which means the growth, initially exponential, is eventually constrained by the carrying capacity \(M\text{.}\) The general solution of (29.1.1) is

\begin{equation} y = \frac{M}{1 + Ae^{-rx}}\label{eq-general-logistic-curve}\tag{29.1.2} \end{equation}

where \(A\) depends on the initial condition. In practice, we do not know the values of \(A, M, r\) but we can try to estimate them by fitting a function of type (29.1.2) to observations \((x_k, y_k)\text{,}\) \(k=1, \dots, n\text{.}\) It seems impossible to transform the data in a way that will make all three parameters linear (logit transformation works only if \(M\) is known).

Recall that least squares method means minimizing the sum of squares of residuals, which is a function of parameters \(A, M, r\text{:}\)

\begin{equation} S(A, M, r) = \sum_{k=1}^n \left(y_k - \frac{M}{1 + Ae^{-rx_k}}\right)^2\label{eq-sum-logistic-residuals}\tag{29.1.3} \end{equation}

Since \(S\) is a smooth function, we could theoretically try the multivariable calculus approach: finds its partial derivatives of first order, equate them to zero, and solve the resulting system of equations:

\begin{equation} \frac{\partial S}{\partial A} = 0, \quad \frac{\partial S}{\partial M} = 0, \quad \frac{\partial S}{\partial r} = 0\label{eq-critical-points-eq}\tag{29.1.4} \end{equation}

But this system will be nonlinear and is unlikely to have a symbolic solution. Should we try some multivariable root-finding method such as Newton's method?

In practice, trying to minimize a function by solving (29.1.4) is not the most productive approach: there are probably many critical points, some of them saddle points, and some of them may be points of local maximum rather than minimum. It is better to search for a minimum directly, which is a problem of multivariable optimization. We will study optimization in the last part of this course. For now, we'll use the built-in Matlab tool fminsearch, the underlying principle of which (the Nelder-Mead method) will be considered later.