Chapter 33 The Nelder-Mead method
Functions of one variable can be efficiently minimized by a combination of golden section and parabolic interpolation methods, which is implemented by fminbnd command in Matlab. These methods are derivative-free: we do not need to either know or estimate any derivatives of the objective function. In contrast, all methods for minimization in several variables that we studied so far require derivatives. The goal of this chapter is to understand a derivative-free minimization method introduced by Nelder and Mead in 1965, which powers the Matlab command fminsearch.