Taylor Series
(Lecture 20 of Mathematical Methods II.)
We will call a function analytic if it given by a convergent power series in an open disc:
$$f(z)=\sum_{n=0}^\infty c_n(z-z_0)^n$$
If $z_0=0$, such a Taylor series is called a Maclaurin series. The evaluation of the coefficients $c_n$ is obtained easily by differentiating term by term:
$$c_n=\frac{f^{(n)}(z_0)}{n!}$$
This makes analytic functions particularly comfortable to work with:
- The function can be approximated by its partial sums, with more accurate results obtained by including more terms of the series.
- Differentiation and integration can be performed term by term and is hence particularly easy.
In the real space, Taylor Series have some disagreeable features. For instance, the function
$$f(x)=\exp(-1/x^2)$$
is infinitely differentiable at $x=0$ but each derivative at this point is zero, therefore the Taylor series does not converge to its function. This is an example of a smooth but non-analytic functions, which have some applications for instance in the theory of distributions.
Let us consider another perplexing case of the real variable. The function:
$$f(x)=\frac{1}{1+x^2}$$
seems to be a nice function: it is everywhere defined, it is bounded and always positive. Its Maclaurin series is tedious but in principle straightforward to derive.
\begin{align} \left.\left(\frac{1}{1+x^2}\right)'\right|_{x=0}&=\left.-\frac{2x}{(1+x^2)^2}\right|_{x=0}=0\\ \left.\left(-\frac{2x}{(1+x^2)^2}\right)'\right|_{x=0}&=\left.\frac{8x^2}{(1+x^2)^3}-\frac{2}{(1+x^2)^2}\right|_{x=0}=-2\\ \left.\left(\frac{1}{1+x^2}\right)'''\right|_{x=0}&=\left.-\frac{48x^3}{(1+x^2)^4}+\frac{24 x}{(1+x^2)^3}\right|_{x=0}=0\,. \end{align}
Next and every other terms will be zero because of the numerator, while nonzero terms progress like alternating factorials. Therefore, the Maclaurin series reads:
$$1-x^2+x^4-x^6+\cdots=\sum_{k=0}^\infty(-1)^kx^{2k}$$
It converges only for $-1<x<1$ since it has two accumulations points ($-1$ and $1$) at $x=\pm 1$ and therefore has no limit.
It seems strange that the series diverges at $x=\pm1$ where the function has no problem (it is equal to $1/2$), or beyond where the function is equally well defined. In fact, one can find a Taylor series that extends beyond the interval $]-1,1[$, for instance by expanding about $x_0=1/2$ (using the previous results):
\begin{equation} \tag{1} f\left(\frac{1}{2}+x\right)=\frac{1}{1+\left(1/2\right)^2}+\left(-\frac{16}{25}\right)x+\left(-\frac{32}{125}\right)\frac{x}{2}+\left(\frac{2304}{625}\right)\frac{x^3}{3!}+\cdots \end{equation}
which provides $f\left(\frac{1}{2}+\frac{1}{2}\right)=\frac{328}{625}\approx0.52$ (next term is 1/2 up to three digits after the comma). One can even compute for some points past the previous limit, e.g., for $x=\frac{1}{2}+\frac{1}{10}$, we would find, for increasing orders of the expansion:
\begin{equation} \tag{2} \{0.8, 0.42, 0.37, 0.50, 0.452, 0.438, 0.459, 0.453, \dots\} \end{equation}
converging towards $1/(1+[1+1/10]^2)\approx0.4524$. The convergence is slow and gets slower until point $\frac{1+\sqrt{5}}{2}$ where it diverges again.
All this is understood by shifting to the complex plane. The Taylor series defines a power series that one can evaluate for complex arguments. For instance:
\begin{equation} \tag{3} \frac{1}{1+(i/2)^2}=\frac{4}{3} \end{equation}
which is indeed the limit of
\begin{equation} \tag{4} 1-(i/2)^2+(i/2)^4-(i/2)^6+\cdots=\sum_{k=0}^\infty1/(2^{2k})=\sum_{k=0}^\infty(1/4)^k=1/(1-1/4)=4/3\,. \end{equation}
It is now apparent what is the problem with the Taylor series on the real axis. The problem is on the imaginary axis, where the function has two poles at $\pm i$. Given the way a power series converges within its radius of convergence, the divergence at~$\pm1$ is required to allow that at $\pm i$. The ability to define a Taylor series around a different centre allows a so-called process of analytic continuation. The limit of the expansion about $1/2$ is simply that defined by the new radius of convergence around this point, which is $|\frac{1}{2}-(\pm i)|=\sqrt{5}/{2}$. One can patch the complex plane with overlapping circles that go round divergences and reconstruct the entire analytic function from its definition on a neighborhood only.
Starting from Cauchy's integral formula on a circle centered on~$z_0$:
\begin{equation} \tag{5} f(z)=\frac{1}{2i\pi}\oint_\mathcal{C}\frac{f(w)}{w-z}\,dw \end{equation}
with~$w\in\mathcal{C}$ and $z$ in the domain circled by it, we decompose:
\begin{align} \tag{6} \frac{1}{w-z}&=\frac{1}{w-z_0+z_0-z}\\ &=\frac{1}{(w-z_0)\left(\displaystyle 1-\frac{z-z_0}{w-z_0}\right)} \end{align}
Since $z$ is inside the circle, defining $q=(z-z_0)/(w-z_0)$, we have $|q|<1$. Using:
\begin{equation} \tag{7} \frac{1}{1-q}=\sum_{k=0}^N q^k+\frac{q^{N+1}}{1-q} \end{equation}
we find:
\begin{equation} \frac{1}{w-z}=\frac{1}{w-z_0}\left[\sum_{k=0}^N\frac{(z-z_0)^k}{(w-z_0)^k}+\frac{1}{w-z}\frac{(z-z_0)^{N+1}}{(w-z_0)^N}\right] \end{equation}
since $1-q=(w-z)/(w-z_0)$. Inserting this back into Cauchy's integral:
\begin{align} \tag{8} f(z)&=\frac{1}{2i\pi}\sum_{k=0}^N(z-z_0)^k\oint_\mathcal{C}\frac{f(w)}{(w-z_0)^{k+1}}dw+ \frac{1}{2i\pi}(z-z_0)^{N+1}\oint_\mathcal{C}\frac{f(w)}{(w-z_0)^{N+1}(w-z)}dw\\ &=\sum_{k=0}^N\frac{f^{(k)}(z_0)}{k!}(z-z_0)^k+R_N(z) \end{align}
with $R_N(z)$ the so-called Taylor remainder:
\begin{equation} \tag{9} R_N(z)=\frac{(z-z_0)^{N+1}}{2i\pi}\oint_\mathcal{C}\frac{f(w)}{(w-z_0)^{N+1}(w-z)}dw\,. \end{equation}
The convergence of the series is assured if $\lim_{n\rightarrow\infty}R_n(z)=0$, which is easily proved: since $z$ is inside the circle delimited by~$\mathcal{C}$ and $w$ is on the circle, $|z-w|>0$. The function being analytic, it is continuous and therefore bounded in $\mathcal{C}$, say we call $M$ the real value such that $|f(w)/(w-z)|<M$ for all~$w\in\mathcal{D}$, then:
\begin{align} \tag{10} |R_N(z)|&=\left|\frac{(z-z_0)^{N+1}}{2i\pi}\oint_\mathcal{C}\frac{f(w)}{(w-z_0)^{N+1}(w-z)}dw\right|\\ &\le\frac{|z-z_0|^{N+1}}{2\pi}\frac{M}{|w-z_0|^{N+1}}2\pi|w-z_0|\\ &\le\left|\frac{z-z_0}{w-z_0}\right|^{N+1}M|w-z_0|\\ \end{align}
which converges to zero as~$N\rightarrow\infty$ since $|z-z_0|<|w-z_0|$ and $|w-z_0|$ is constant. This shows that the Taylor series can be made arbitrarily close to the function.
Calculation of Taylor series are usually made on the basis of a handful of particular cases that should be known, through combinations, substitutions and other tricks like integration and derivation. In other cases, or if unsure, one can also use the definition. For instance, for :
\begin{equation} \tag{11} f(z)=\frac{1}{1-z}\quad\Rightarrow\quad f^{(n)}(z)=\frac{n!}{(1-z)^{n+1}}\quad\Rightarrow\quad f^{(n)}(0)=n! \end{equation}
and consequently:
\begin{equation} \tag{12} \frac{1}{1-z}=\sum_{k=0}^\infty z^k \end{equation}
The exponential is similarly easily calculated, since $\exp(z)'=\exp(z)$, $\exp(z)=\sum_{k=0}^\infty z^k/k!$.
The series $f(z)=(1+z)^\nu$ is also often used and easily derived:
\begin{align} f'(z)&=\nu(1+z)^{\nu-1}\\ f''(z)&=\nu(\nu-1)(1+z)^{\nu-2}\\ \vdots\nonumber\\ f^{k}(z)&=\nu(\nu-1)\cdots(\nu-k+1)(1+z)^{\nu-k} \end{align}
The Maclaurin series is immediate:
\begin{equation} \tag{13} f(z)=\sum_{k=0}^\infty\frac{f^{k}(0)}{k!}z^k=\sum_{k=0}^\infty{\nu\choose k}z^k \end{equation}
where we have defined the generalized binomial coefficients:
\begin{equation} \tag{14} {\nu\choose k}=\frac{\nu(\nu-1)(\nu-2)\cdots(\nu-k+1)}{k!}\,. \end{equation}
For this reason, this series is called the Binomial series. It is infinite unless~$\nu$ is an integer, in which case the binomial ultimately cancel. By the ratio test, the ratio of consecutive terms is found to be:
\begin{align} \tag{15} |a_{k+1}/a_k|&=\left|\frac{\nu(\nu-1)(\nu-2)\cdots(\nu-k)}{(k+1)!}\frac{k!}{\nu(\nu-1)(\nu-2)\cdots(\nu-k+1)}\right||z|\\ &=\left|\frac{\nu-k}{k+1}\right||z|=\left|\frac{\nu/k-1}{1+1/k}\right||z| \end{align} % which tends to~$|z|$ when~$k\rightarrow\infty$. Therefore the Binomial series converges for~$|z|<1$ an diverges for~$|z|\ge1$.
From this, we can compute, for instance, $\sqrt{1+z}=1+z/2-z^2/8+z^3/16+\cdots$ or $1/(1+z)^m=1-mz+m(m+1)z^2/2-m(m+1)(m+2)z^3/6+\cdots$, etc.
The series $1/(1+z^2)$ which we have considered at length previously arises straightforwardly from the geometric series by substitution $z\rightarrow -z^2$.
Another example of evaluation by substitution. Say we want the Taylor series of $\sin(z^2)$. We know that $\sin(z)=\sum_{k=0}^\infty(-1)^k\frac{z^{2k+1}}{(2k+1)!}$, therefore, $\sin(z^2)=\sum_{k=0}^\infty(-1)^k\frac{z^{4k+2}}{(2k+1)!}=z^2-z^6/3!+z^{10}/5!-\cdots$
A calculation by integration: The Maclaurin series of $f(z)=\arctan(z)$ can be found from the fact that $f'(z)=1/(1+z^2)=\sum_{k=0}^\infty(-1)^kz^{2k}$. Integrating termwise:
\begin{equation} \tag{16} \arctan(z)=\sum_{k=0}^\infty\frac{(-1)^k}{2k+1}z^{2k+1}=z-\frac{z^3}{3}+\frac{z^5}{5}-\cdots \end{equation}
