Processing math: 100%
Skip to main content
Numerical Methods with Programming
Leonid Kovalev
Contents
Prev
Up
Next
Contents
Prev
Up
Next
Front Matter
Acknowledgements
Preface
I
Using Matlab
1
Scalars, vectors and matrices
Matlab interface
Assigning scalar values
Creating vectors and matrices
Operations on vectors and matrices
Accessing the entries of vectors and matrices
Comments and sections in a script
Examples and questions
Homework
2
Linear systems, array operations, plots
Systems of linear equations
More tools for constructing matrices
Array operations
Implicit expansion of arrays
Plotting
Examples and questions
Homework
3
Built-in functions, input, output
Built-in functions
Random numbers and histograms
User input
Formatted output
Examples and questions
Homework
4
Control flow: loops and conditional statements
for
loop
while
loop
if
statement
Breaking out of a loop
Measuring the computation time
Examples and questions
Homework
5
User-defined functions
Anonymous functions
Named functions
Examples and questions
Homework
II
Solving Equations
6
Systems of linear equations
Classification of linear systems
Systems without free variables
Systems with free variables
The effect of round-off errors
Examples and questions
Homework
7
Root finding: bisection
Motivation for solving equations numerically
The meaning of a numeric solution
Bisection method
Limitations of the bisection method
Examples and questions
Homework
8
Root finding: fixed point iteration
Classification of fixed points
Rewriting equations in the fixed-point form
The speed of convergence of fixed-point iteration
Examples and questions
Homework
9
Newton's method and its relatives
Newton's method
Potential issues with Newton's method
The secant method
How
fzero
works
The relaxation parameter
Examples and questions
Homework
10
Systems of several nonlinear equations: multivariate Newton's method
Systems of nonlinear equations
Multivariate Newton's method
Potential issues
Examples and questions
Homework
11
Broyden's method
Idea of Broyden's method
Outer products
Details of Broyden's method
Examples and questions
Homework
III
Numerical Calculus
12
Numerical differentiation
Definition of derivative and the order of error
Loss of significance
Symmetric difference formulas
Richardson extrapolation
Examples and questions
Homework
13
Numerical integration: basic rules
Riemann sums
Left, right, midpoint rules
Trapezoidal rule
Error estimates
Examples and questions
Homework
14
Simpson's rule and other Newton-Cotes rules
Simpson's rule
Other derivations of Simpson's rule
Newton-Cotes rules
Examples and questions
Homework
15
Legendre polynomials and Laguerre polynomials
Motivation: search for better evaluation points
Legendre polynomials
Laguerre polynomials
Examples and questions
Homework
16
Gauss-Legendre and Gauss-Laguerre integration
Brief recap of orthogonal polynomials
Gauss-Legendre integration
Gauss-Laguerre integration
Examples and questions
Homework
17
Adaptive integration
Why we need adaptive methods
Estimating error by using two step sizes
Recursive subdivision algorithm
Combining two rules
Examples and questions
Homework
18
Multivariable integration
Double integrals over rectangles
Double integrals over general regions
High-dimensional integration: Monte-Carlo method
Homework
19
Differential equations: Euler's method and its relatives
Ordinary differential equations
Euler's method
Estimating the accuracy of numeric ODE methods
Improving Euler's method
Solving systems of differential equations
Homework
20
Modeling with differential equations
Models of an epidemic
Predator-prey models
Examples and questions
Homework
IV
Modeling Data
21
Polynomial Interpolation
Interpolation in the monomial basis
Lagrange interpolating polynomial
Newton interpolating polynomial
Homework
22
Chebyshev Polynomials and Interpolation
Estimating the error of polynomial interpolation
Chebyshev polynomials and interpolation
Chebyshev extreme points
Homework
23
Spline Interpolation
Piecewise linear interpolation
Cubic splines
Using cubic splines
Homework
24
Spline Approximation
Weak points of spline interpolation
Hat functions and B-splines
Construction of cardinal B-splines
Homework
25
Discrete Fourier Transform
Periodic functions and trigonometric polynomials
Fourier series
Discrete Fourier Transform
Trigonometric Interpolation
Homework
26
Applications of Discrete Fourier Transform
Working with images in Matlab
Two-dimensional DFT
Cosine interpolation of non-periodic functions
Homework
27
Linear Least Squares
Overview of least squares
Overfitting, training and testing
The standard error of parameters
Multiple regression
Homework
28
Transforming Data for LLS
Exponential fit
Logistic fit
Weighted least squares
Fitting an implicit equation
Homework
29
Nonlinear Least Squares
Motivating examples for Nonlinear Least Squares
Using fminsearch for curve-fitting
Beyond curve-fitting: source location
Homework
V
Optimization
30
Single Variable Minimization
Basic concepts of optimization
Brute force search
Newton method for minimization
Golden section search
Homework
31
Parabolic Interpolation and Gradient Descent
Successive parabolic interpolation
How
fminbnd
works
Gradient descent in one dimension
Homework
32
Gradient methods and Newton's method
Gradient descent in several variables
Newton's method for multivariate minimization
Conjugate gradient method
Homework
33
The Nelder-Mead method
First attempt at derivative-free minimization
Reflection-contraction combination
Reflection-contraction-expansion Nelder-Mead method
Nelder-Mead method in higher dimensions
Homework
34
Constrained Optimization
Penalty method
Lagrange multiplier method
Homework
35
Linear Programming
Introduction to linear programming
Duality in linear programming
Interpretation of duality in microeconomics
Duality in optimal transportation
Homework
36
Classification Problems
Classification and logistic regression
Classification and linear programming
Homework
Authored in PreTeXt
🔗
Numerical Methods with Programming
Leonid Kovalev
Department of Mathematics
Syracuse University
lvkovale@syr.edu
December 8, 2021
Acknowledgements
Preface