When deriving a finite difference approximation of the
j
{\displaystyle j}
th derivative of a function
f
:
R
→
R
{\displaystyle f:\mathbb {R} \rightarrow \mathbb {R} }
, we wish to find
a
1
,
a
2
,
.
.
.
,
a
n
∈
R
{\displaystyle a_{1},a_{2},...,a_{n}\in \mathbb {R} }
and
b
1
,
b
2
,
.
.
.
,
b
n
∈
R
{\displaystyle b_{1},b_{2},...,b_{n}\in \mathbb {R} }
such that
f
(
j
)
(
x
0
)
=
h
−
j
∑
i
=
1
n
a
i
f
(
x
0
+
b
i
h
)
+
O
(
h
k
)
as
h
→
0
{\displaystyle f^{(j)}(x_{0})=h^{-j}\sum _{i=1}^{n}a_{i}f(x_{0}+b_{i}h)+O(h^{k}){\text{ as }}h\to 0}
or, equivalently,
h
−
j
∑
i
=
1
n
a
i
f
(
x
0
+
b
i
h
)
=
f
(
j
)
(
x
0
)
+
O
(
h
k
)
as
h
→
0
{\displaystyle h^{-j}\sum _{i=1}^{n}a_{i}f(x_{0}+b_{i}h)=f^{(j)}(x_{0})+O(h^{k}){\text{ as }}h\to 0}
where
O
(
h
k
)
{\displaystyle O(h^{k})}
is the error, the difference between the correct answer and the approximation, expressed using Big-O notation . Because
h
{\displaystyle h}
may be presumed to be small, a larger value for
k
{\displaystyle k}
is better than a smaller value.
A general method for finding the coefficients is to generate the Taylor expansion of
h
−
j
∑
i
=
1
n
a
i
f
(
x
0
+
b
i
h
)
{\displaystyle h^{-j}\sum _{i=1}^{n}a_{i}f(x_{0}+b_{i}h)}
and choose
a
1
,
a
2
,
.
.
.
,
a
n
{\displaystyle a_{1},a_{2},...,a_{n}}
and
b
1
,
b
2
,
.
.
.
,
b
n
{\displaystyle b_{1},b_{2},...,b_{n}}
such that
f
(
j
)
(
x
0
)
{\displaystyle f^{(j)}(x_{0})}
and the remainder term are the only non-zero terms. If there are no such coefficients, a smaller value for
k
{\displaystyle k}
must be chosen.
For a function of
m
{\displaystyle m}
variables
g
:
R
m
→
R
{\displaystyle g:\mathbb {R} ^{m}\rightarrow \mathbb {R} }
, the procedure is similar, except
x
0
,
b
1
,
b
2
,
.
.
.
,
b
n
{\displaystyle x_{0},b_{1},b_{2},...,b_{n}}
are replaced by points in
R
m
{\displaystyle \mathbb {R} ^{m}}
and the multivariate extension of Taylor's theorem is used.
In all single-variable examples,
x
0
∈
R
{\displaystyle x_{0}\in \mathbb {R} }
and
f
:
R
→
R
{\displaystyle f:\mathbb {R} \rightarrow \mathbb {R} }
are unknown, and
h
∈
R
{\displaystyle h\in \mathbb {R} }
is small. Additionally, let
f
{\displaystyle f}
be 5 times continuously differentiable on
R
{\displaystyle \mathbb {R} }
.
Find
a
,
b
,
c
∈
R
{\displaystyle a,b,c\in \mathbb {R} }
such that
a
f
(
x
0
+
h
)
+
b
f
(
x
0
+
c
h
)
h
{\displaystyle {\frac {af(x_{0}+h)+bf(x_{0}+ch)}{h}}}
best approximates
f
′
(
x
0
)
{\displaystyle f'(x_{0})}
.
Let
f
:
R
→
R
{\displaystyle f:\mathbb {R} \rightarrow \mathbb {R} }
be 42 times continuously differentiable on
R
{\displaystyle \mathbb {R} }
. Find the largest
n
∈
N
{\displaystyle n\in \mathbb {N} }
such that
d
f
d
x
(
x
0
)
=
−
f
(
x
0
+
2
h
)
+
8
f
(
x
0
+
h
,
y
0
)
−
8
f
(
x
0
−
h
)
+
f
(
x
0
−
2
h
)
12
h
+
O
(
h
n
)
as
h
→
0
{\displaystyle {\frac {df}{dx}}(x_{0})={\frac {-f(x_{0}+2h)+8f(x_{0}+h,y_{0})-8f(x_{0}-h)+f(x_{0}-2h)}{12h}}+O(h^{n}){\text{ as }}h\to 0}
In other words, find the order of the error of the method.
Solution:
The Taylor expansion of the method is
−
f
(
x
0
+
2
h
)
+
8
f
(
x
0
+
h
,
y
0
)
−
8
f
(
x
0
−
h
)
+
f
(
x
0
−
2
h
)
12
h
=
−
1
+
8
−
8
+
1
12
h
f
(
x
0
)
+
−
2
+
8
+
8
−
2
12
f
′
(
x
0
)
+
h
(
−
4
+
8
−
8
+
4
)
24
f
″
(
x
0
)
+
h
2
(
−
8
+
8
+
8
−
8
72
f
‴
(
x
0
)
+
h
3
(
−
16
+
8
−
8
+
16
288
f
(
4
)
(
x
0
)
+
h
4
(
−
32
+
8
+
8
−
32
)
1440
f
(
4
)
(
x
0
)
+
O
(
h
5
)
as
h
→
0
.
{\displaystyle {\begin{aligned}{\frac {-f(x_{0}+2h)+8f(x_{0}+h,y_{0})-8f(x_{0}-h)+f(x_{0}-2h)}{12h}}&={\frac {-1+8-8+1}{12h}}f(x_{0})+{\frac {-2+8+8-2}{12}}f'(x_{0})\\&+h{\frac {(-4+8-8+4)}{24}}f''(x_{0})+h^{2}{\frac {(-8+8+8-8}{72}}f'''(x_{0})\\&+h^{3}{\frac {(-16+8-8+16}{288}}f^{(4)}(x_{0})+h^{4}{\frac {(-32+8+8-32)}{1440}}f^{(4)}(x_{0})+O(h^{5}){\text{ as }}h\to 0\,.\end{aligned}}}
Simplifying this algebraically gives
−
f
(
x
0
+
2
h
)
+
8
f
(
x
0
+
h
,
y
0
)
−
8
f
(
x
0
−
h
)
+
f
(
x
0
−
2
h
)
12
h
=
f
′
(
x
0
)
+
h
4
−
30
f
(
5
)
(
x
0
)
+
O
(
h
5
)
as
h
→
0
.
{\displaystyle {\frac {-f(x_{0}+2h)+8f(x_{0}+h,y_{0})-8f(x_{0}-h)+f(x_{0}-2h)}{12h}}=f'(x_{0})+{\frac {h^{4}}{-30}}f^{(5)}(x_{0})+O(h^{5}){\text{ as }}h\to 0\,.}
The multiple of
h
4
{\displaystyle h^{4}}
cannot be removed, so
n
=
4
{\displaystyle n=4}
and by properties of Big-O notation,
−
f
(
x
0
+
2
h
)
+
8
f
(
x
0
+
h
,
y
0
)
−
8
f
(
x
0
−
h
)
+
f
(
x
0
−
2
h
)
12
h
=
f
′
(
x
0
)
+
O
(
h
4
)
as
h
→
0
.
{\displaystyle {\frac {-f(x_{0}+2h)+8f(x_{0}+h,y_{0})-8f(x_{0}-h)+f(x_{0}-2h)}{12h}}=f'(x_{0})+O(h^{4}){\text{ as }}h\to 0\,.}
Find
a
,
b
,
c
∈
R
{\displaystyle a,b,c\in \mathbb {R} }
such that
a
f
(
x
0
−
h
)
+
b
f
(
x
0
)
+
c
f
(
x
0
+
h
)
h
2
{\displaystyle {\frac {af(x_{0}-h)+bf(x_{0})+cf(x_{0}+h)}{h^{2}}}}
best approximates
f
″
(
x
0
)
{\displaystyle f''(x_{0})}
.
In all two-variable examples,
x
0
,
y
0
∈
R
{\displaystyle x_{0},y_{0}\in \mathbb {R} }
and
f
:
R
2
→
R
{\displaystyle f:\mathbb {R} ^{2}\rightarrow \mathbb {R} }
are unknown, and
h
∈
R
{\displaystyle h\in \mathbb {R} }
is small.
Non-Mixed Derivatives
edit
Because of the nature of partial derivatives , some of them may be calculated using single-variable methods. This is done by holding constant all but one variable to form a new function of one variable. For example if
g
y
(
y
)
=
f
(
x
0
,
y
)
{\displaystyle g_{y}(y)=f(x_{0},y)}
, then
d
f
d
y
(
x
0
,
y
0
)
=
d
g
d
y
(
y
0
)
{\displaystyle {\frac {df}{dy}}(x_{0},y_{0})={\frac {dg}{dy}}(y_{0})}
.
Find an approximation of
d
d
y
f
(
x
0
,
y
0
)
{\displaystyle {\frac {d}{dy}}f(x_{0},y_{0})}
Solution:
Because we are differentiating with respect to only one variable, we can hold x constant and use the result of one of the single-variable examples:
f
(
x
0
,
y
0
+
h
)
−
f
(
x
0
,
y
0
−
h
)
2
h
=
d
f
d
y
(
x
0
,
y
0
)
+
O
(
h
2
)
as
h
→
0
{\displaystyle {\frac {f(x_{0},y_{0}+h)-f(x_{0},y_{0}-h)}{2h}}={\frac {df}{dy}}(x_{0},y_{0})+O(h^{2}){\text{ as }}h\to 0}
Mixed derivatives may require the multivariate extension of Taylor's theorem .
Let
f
:
R
2
→
R
{\displaystyle f:\mathbb {R} ^{2}\rightarrow \mathbb {R} }
be 42 times continuously differentiable on
R
2
{\displaystyle \mathbb {R} ^{2}}
and let
g
:
R
3
→
R
{\displaystyle g:\mathbb {R} ^{3}\rightarrow \mathbb {R} }
be defined as
g
(
x
0
,
y
0
,
h
)
=
f
(
x
0
+
h
,
y
0
+
h
)
+
f
(
x
0
−
h
,
y
0
−
h
)
−
f
(
x
0
−
h
,
y
0
+
h
)
−
f
(
x
0
+
h
,
y
0
−
h
)
4
h
2
.
{\displaystyle g(x_{0},y_{0},h)={\frac {f(x_{0}+h,y_{0}+h)+f(x_{0}-h,y_{0}-h)-f(x_{0}-h,y_{0}+h)-f(x_{0}+h,y_{0}-h)}{4h^{2}}}\,.}
Find the largest
n
∈
N
{\displaystyle n\in \mathbb {N} }
such that
d
2
f
d
x
d
y
(
x
0
,
y
0
)
=
g
(
x
0
,
y
0
,
h
)
+
O
(
h
n
)
as
h
→
0
.
{\displaystyle {\frac {d^{2}f}{dxdy}}(x_{0},y_{0})=g(x_{0},y_{0},h)+O(h^{n}){\text{ as }}h\to 0\,.}
In other words, find the order of the error of the approximation.
Solution:
The first few terms of the multivariate Taylor expansion of
f
{\displaystyle f}
around
(
x
0
,
y
0
)
{\displaystyle (x_{0},y_{0})}
are
f
(
x
0
+
x
,
y
0
+
y
)
=
f
(
x
0
,
y
0
)
+
x
d
f
d
x
(
x
0
,
y
0
)
+
y
d
f
d
y
(
x
0
,
y
0
)
+
x
2
2
d
2
f
d
x
2
(
x
0
,
y
0
)
+
x
y
d
d
y
d
f
d
x
(
x
0
,
y
0
)
+
y
2
2
d
2
f
d
y
2
(
x
0
,
y
0
)
+
x
3
6
d
3
f
d
x
3
(
x
0
,
y
0
)
+
x
2
y
2
d
d
y
d
2
f
d
x
2
(
x
0
,
y
0
)
+
x
y
2
2
d
2
d
y
2
d
f
d
x
(
x
0
,
y
0
)
+
y
3
6
d
3
f
d
y
3
(
x
0
,
y
0
)
+
x
4
24
d
4
f
d
x
4
(
x
0
,
y
0
)
+
x
3
y
6
d
d
y
d
3
f
d
x
3
(
x
0
,
y
0
)
+
x
2
y
2
4
d
2
d
y
2
d
2
f
d
x
2
(
x
0
,
y
0
)
+
x
y
3
6
d
3
d
y
3
d
f
d
x
(
x
0
,
y
0
)
+
y
4
24
d
4
f
d
y
4
(
x
0
,
y
0
)
+
O
(
x
5
)
+
O
(
y
5
)
as
x
,
y
→
0
.
{\displaystyle {\begin{aligned}f(x_{0}+x,y_{0}+y)&=f(x_{0},y_{0})\\&+x{\frac {df}{dx}}(x_{0},y_{0})+y{\frac {df}{dy}}(x_{0},y_{0})\\&+{\frac {x^{2}}{2}}{\frac {d^{2}f}{dx^{2}}}(x_{0},y_{0})+xy{\frac {d}{dy}}{\frac {df}{dx}}(x_{0},y_{0})+{\frac {y^{2}}{2}}{\frac {d^{2}f}{dy^{2}}}(x_{0},y_{0})\\&+{\frac {x^{3}}{6}}{\frac {d^{3}f}{dx^{3}}}(x_{0},y_{0})+{\frac {x^{2}y}{2}}{\frac {d}{dy}}{\frac {d^{2}f}{dx^{2}}}(x_{0},y_{0})+{\frac {xy^{2}}{2}}{\frac {d^{2}}{dy^{2}}}{\frac {df}{dx}}(x_{0},y_{0})+{\frac {y^{3}}{6}}{\frac {d^{3}f}{dy^{3}}}(x_{0},y_{0})\\&+{\frac {x^{4}}{24}}{\frac {d^{4}f}{dx^{4}}}(x_{0},y_{0})+{\frac {x^{3}y}{6}}{\frac {d}{dy}}{\frac {d^{3}f}{dx^{3}}}(x_{0},y_{0})+{\frac {x^{2}y^{2}}{4}}{\frac {d^{2}}{dy^{2}}}{\frac {d^{2}f}{dx^{2}}}(x_{0},y_{0})+{\frac {xy^{3}}{6}}{\frac {d^{3}}{dy^{3}}}{\frac {df}{dx}}(x_{0},y_{0})+{\frac {y^{4}}{24}}{\frac {d^{4}f}{dy^{4}}}(x_{0},y_{0})\\&+O(x^{5})+O(y^{5}){\text{ as }}x,y\to 0\,.\end{aligned}}}
We substitute the expansion for
f
{\displaystyle f}
into the approximation
g
{\displaystyle g}
to obtain
g
(
x
0
,
y
0
,
h
)
=
1
+
1
−
1
−
1
4
h
2
f
(
x
0
,
y
0
)
+
1
−
1
+
1
−
1
4
h
d
f
d
x
(
x
0
,
y
0
)
+
1
−
1
−
1
+
1
4
h
d
f
d
y
(
x
0
,
y
0
)
+
1
+
1
−
1
−
1
8
d
2
f
d
x
2
(
x
0
,
y
0
)
+
1
+
1
+
1
+
1
4
d
d
y
d
f
d
x
(
x
0
,
y
0
)
+
1
+
1
−
1
−
1
8
d
2
f
d
y
2
(
x
0
,
y
0
)
+
h
(
1
−
1
+
1
−
1
24
d
3
f
d
x
3
(
x
0
,
y
0
)
+
1
−
1
−
1
+
1
8
d
d
y
d
2
f
d
x
2
(
x
0
,
y
0
)
+
1
−
1
+
1
−
1
8
d
2
d
y
2
d
f
d
x
(
x
0
,
y
0
)
+
1
−
1
−
1
+
1
24
d
3
f
d
y
3
(
x
0
,
y
0
)
)
+
h
2
(
1
+
1
−
1
−
1
96
d
4
f
d
x
4
(
x
0
,
y
0
)
+
1
+
1
+
1
+
1
24
d
d
y
d
3
f
d
x
3
(
x
0
,
y
0
)
+
1
+
1
−
1
−
1
16
d
2
d
y
2
d
2
f
d
x
2
(
x
0
,
y
0
)
+
1
+
1
+
1
+
1
24
d
3
d
y
3
d
f
d
x
(
x
0
,
y
0
)
+
1
+
1
−
1
−
1
96
d
4
f
d
y
4
(
x
0
,
y
0
)
)
+
O
(
h
3
)
as
h
→
0
.
{\displaystyle {\begin{aligned}g(x_{0},y_{0},h)&={\frac {1+1-1-1}{4h^{2}}}f(x_{0},y_{0})\\&+{\frac {1-1+1-1}{4h}}{\frac {df}{dx}}(x_{0},y_{0})+{\frac {1-1-1+1}{4h}}{\frac {df}{dy}}(x_{0},y_{0})\\&+{\frac {1+1-1-1}{8}}{\frac {d^{2}f}{dx^{2}}}(x_{0},y_{0})+{\frac {1+1+1+1}{4}}{\frac {d}{dy}}{\frac {df}{dx}}(x_{0},y_{0})+{\frac {1+1-1-1}{8}}{\frac {d^{2}f}{dy^{2}}}(x_{0},y_{0})\\&+h\left({\frac {1-1+1-1}{24}}{\frac {d^{3}f}{dx^{3}}}(x_{0},y_{0})+{\frac {1-1-1+1}{8}}{\frac {d}{dy}}{\frac {d^{2}f}{dx^{2}}}(x_{0},y_{0})+{\frac {1-1+1-1}{8}}{\frac {d^{2}}{dy^{2}}}{\frac {df}{dx}}(x_{0},y_{0})+{\frac {1-1-1+1}{24}}{\frac {d^{3}f}{dy^{3}}}(x_{0},y_{0})\right)\\&+h^{2}\left({\frac {1+1-1-1}{96}}{\frac {d^{4}f}{dx^{4}}}(x_{0},y_{0})+{\frac {1+1+1+1}{24}}{\frac {d}{dy}}{\frac {d^{3}f}{dx^{3}}}(x_{0},y_{0})+{\frac {1+1-1-1}{16}}{\frac {d^{2}}{dy^{2}}}{\frac {d^{2}f}{dx^{2}}}(x_{0},y_{0})+{\frac {1+1+1+1}{24}}{\frac {d^{3}}{dy^{3}}}{\frac {df}{dx}}(x_{0},y_{0})+{\frac {1+1-1-1}{96}}{\frac {d^{4}f}{dy^{4}}}(x_{0},y_{0})\right)\\&+O(h^{3}){\text{ as }}h\to 0\,.\end{aligned}}}
Because of the careful choices of coefficients, we can simplify this to
g
(
x
0
,
y
0
,
h
)
=
d
d
y
d
f
d
x
(
x
0
,
y
0
)
+
h
2
6
(
d
d
y
d
3
f
d
x
3
(
x
0
,
y
0
)
+
d
3
d
y
3
d
f
d
x
(
x
0
,
y
0
)
)
+
O
(
h
3
)
as
h
→
0
.
{\displaystyle g(x_{0},y_{0},h)={\frac {d}{dy}}{\frac {df}{dx}}(x_{0},y_{0})+{\frac {h^{2}}{6}}\left({\frac {d}{dy}}{\frac {d^{3}f}{dx^{3}}}(x_{0},y_{0})+{\frac {d^{3}}{dy^{3}}}{\frac {df}{dx}}(x_{0},y_{0})\right)+O(h^{3}){\text{ as }}h\to 0\,.}
We note that Big-O notation permits us to write the last 3 terms as
O
(
h
2
)
as
h
→
0
{\displaystyle O(h^{2}){\text{ as }}h\to 0}
. Thus,
g
(
x
0
,
y
0
,
h
)
=
d
d
y
d
f
d
x
(
x
0
,
y
0
)
+
O
(
h
2
)
as
h
→
0
.
{\displaystyle g(x_{0},y_{0},h)={\frac {d}{dy}}{\frac {df}{dx}}(x_{0},y_{0})+O(h^{2}){\text{ as }}h\to 0\,.}
Because the multiples of
h
2
{\displaystyle h^{2}}
are unaffected by adding more terms to the Taylor expansion,
n
=
2
{\displaystyle n=2}
is the greatest natural number satisfying the conditions given in the problem.
Implementing these methods is reasonably simple in programming languages that support higher-order functions . For example, the method from the first example may be implemented in C++ using function pointers , as follows:
// Returns an approximation of the derivative of f at x.
double derivative ( double ( * f )( double ), double x , double h = 0.01 ) {
return ( f ( x + h ) - f ( x - h )) / ( 2 * h );
}