m (Created page with "{{wl1}} = The Wiener-Khinchin theorem = The '''''Wiener-Khinchin''''' theorem is one of the central results of statistical and stochastic physics and of signal processing. It...") |
m |
||
| Line 7: | Line 7: | ||
To appreciate the importance of this result, one needs to first understand what is a power spectrum, what is an autocorrelation function and why their connection as established by the theorem is, beyond useful for its practical purpose, insightful | To appreciate the importance of this result, one needs to first understand what is a power spectrum, what is an autocorrelation function and why their connection as established by the theorem is, beyond useful for its practical purpose, insightful | ||
| − | The power spectrum $S(\omega)$ is the distribution of frequency or energy of a system, which has $a(t)$. I have used here notations of [[quantum optics]] for a quantized field with annihilation operator $a$. | + | <!--The power spectrum $S(\omega)$ is the distribution of frequency or energy of a system, which has $a(t)$. I have used here notations of [[quantum optics]] for a quantized field with annihilation operator $a$. Would the signal have a [[Fourier transform]], one would --> |
The idea was first proposed by [[Einstein]] at a meeting of the [[Swiss]] "Société de Physique" whose compte-rendu lists the people who first heard about this connection (as well as what they discussed about themselves) | The idea was first proposed by [[Einstein]] at a meeting of the [[Swiss]] "Société de Physique" whose compte-rendu lists the people who first heard about this connection (as well as what they discussed about themselves) | ||
| Line 21: | Line 21: | ||
Einstein drafted a longer version, that was, however, never published (it is available as [https://einsteinpapers.press.princeton.edu/vol4-trans/314 Doc. 30 of the 4th volume of his collected papers] (the two-page memo appears there as the previous [https://einsteinpapers.press.princeton.edu/vol4-trans/312 Doc. 29]). | Einstein drafted a longer version, that was, however, never published (it is available as [https://einsteinpapers.press.princeton.edu/vol4-trans/314 Doc. 30 of the 4th volume of his collected papers] (the two-page memo appears there as the previous [https://einsteinpapers.press.princeton.edu/vol4-trans/312 Doc. 29]). | ||
| − | The result comes in full bloom with [[Norbert Wiener]], who, on opposite, writes almost a book on the topic of "Generalized harmonic analysis"{{cite|wiener30a}}. | + | The result comes in full bloom with [[Norbert Wiener]], who, on opposite, writes almost a book (a 142 pages paper) on the topic of "Generalized harmonic analysis" {{cite|wiener30a}}. |
| + | |||
| + | A brief account of the main results is that energy typically comes as the square of the signal, i.e., | ||
| + | |||
| + | $$E=\int_{-\infty}^\infty|V(t)|^2\,dt$$ | ||
| + | |||
| + | for the total energy, while in the frequency space, from Plancherel's theorem, the same relation holds for the Fourier transform: | ||
| + | |||
| + | $$E=\int_{-\infty}^\infty|\tilde V(\omega)|^2\,dt$$ | ||
| + | |||
| + | from which one can infer that $|\tilde V(\omega)|^2$ is the energy | ||
| + | density, i.e., $|\tilde V(\omega)|^2\Delta\omega$ is how much energy | ||
| + | the system has in the window $[\omega, \omega+\Delta\omega]$. | ||
| + | |||
| + | However, a stationary signal does not have, in general, a Fourier transform, but a truncated version $V_T$ does: | ||
| + | |||
| + | $$V_T(t)\equiv V(t)\mathbb{1}_{\{|t|\le T\}}$$ | ||
| + | |||
| + | so that now one can define the FT $\tilde V(\omega)$: | ||
| + | |||
| + | $$ V_T(t)={1\over2\pi}\int_{-\infty}^\infty\tilde V_T(\omega)e^{i\omega t}\,d\omega$$ | ||
| + | |||
| + | and while the following definition would match the initial association of a power spectrum to the signal itself: | ||
| + | |||
| + | $$\lim_{T\to\infty}{|\tilde V_T(\omega)|^2\over 2T}$$ | ||
| + | |||
| + | where the limit is taken to get rid of the truncation, it turns out that such a quantity (known as the [https://en.wikipedia.org/wiki/Periodogram periodogram] has weak convergence properties. However, an ensemble average turns out to be convergent again and provides a good or at least well-behaved definition for the power spectrum: | ||
| + | |||
| + | $$S(\omega)=\lim_{T\to\infty}\left\langle{|\tilde V_T(\omega)|^2\over 2T}\right\rangle\,.$$ | ||
| + | |||
| + | The main contribution from Wiener was to prove that the Fourier transform of the autocorrelation function is equal to this mathematically sound definition of the power spectrum | ||
| + | |||
| + | $$\lim_{T\to\infty}\left\langle{|\tilde V_T(\omega)|^2\over 2T}\right\rangle=\int_{-\infty}^\infty\langle V(t)^*V(t+\tau)\rangle e^{i\omega\tau}\,dt$$ | ||
| + | |||
| + | Thereby, from the previous statements, providing the following definition for the power spectrum, namely, that initially suggested (without proof) by Einstein: | ||
| + | |||
| + | $$S(\omega)=\int_{-\infty}^\infty\langle V(t)^*V(t+\tau)\rangle e^{i\omega\tau}\,dt$$ | ||
| + | |||
| + | The contribution of Khinchin was to extend the domain of validity to stochastic signals, as Wiener restrained his consideration to deterministic ones. | ||
== References == | == References == | ||
Revision as of 14:32, 2 May 2022
The Wiener-Khinchin theorem
The Wiener-Khinchin theorem is one of the central results of statistical and stochastic physics and of signal processing. It states that the power spectrum $S(\omega)$ and the autocorrelation function $\langle\ud{a}(t+\tau)a(\tau)\rangle$ form a Fourier transform pair.
To appreciate the importance of this result, one needs to first understand what is a power spectrum, what is an autocorrelation function and why their connection as established by the theorem is, beyond useful for its practical purpose, insightful
The idea was first proposed by Einstein at a meeting of the Swiss "Société de Physique" whose compte-rendu lists the people who first heard about this connection (as well as what they discussed about themselves)
The communication was saved for posterity in a two-pages memo[1], written in French. Neither Einstein nor other—several prestigious—scientists seem to have appreciated or understood the significance of the statement at the time. Einstein himself discusses at length the technical feasibility of the integration, for which he consigns that his friend M. P. Habicht (not Conrad?) assured him:
M. P. Habicht, m'a montré en outre que la détermination des moyennes de (1) peuvent se faire aisément à l'aide d'un intégrateur mécanique de maniement facile.
One comment is recorded in the minutes of the meeting from the president Pierre Weiss who, despite his input to mean-field theory, also lacks a proper valuation of the result:
M. Weiss fait remarquer combien la méthode de M. Einstein rendra de services à la météorolog-ie très riche en matériaux qui, jusqu'à présent, étaient à peu près inutilisables.
Einstein drafted a longer version, that was, however, never published (it is available as Doc. 30 of the 4th volume of his collected papers (the two-page memo appears there as the previous Doc. 29).
The result comes in full bloom with Norbert Wiener, who, on opposite, writes almost a book (a 142 pages paper) on the topic of "Generalized harmonic analysis" [2].
A brief account of the main results is that energy typically comes as the square of the signal, i.e.,
$$E=\int_{-\infty}^\infty|V(t)|^2\,dt$$
for the total energy, while in the frequency space, from Plancherel's theorem, the same relation holds for the Fourier transform:
$$E=\int_{-\infty}^\infty|\tilde V(\omega)|^2\,dt$$
from which one can infer that $|\tilde V(\omega)|^2$ is the energy density, i.e., $|\tilde V(\omega)|^2\Delta\omega$ is how much energy the system has in the window $[\omega, \omega+\Delta\omega]$.
However, a stationary signal does not have, in general, a Fourier transform, but a truncated version $V_T$ does:
$$V_T(t)\equiv V(t)\mathbb{1}_{\{|t|\le T\}}$$
so that now one can define the FT $\tilde V(\omega)$:
$$ V_T(t)={1\over2\pi}\int_{-\infty}^\infty\tilde V_T(\omega)e^{i\omega t}\,d\omega$$
and while the following definition would match the initial association of a power spectrum to the signal itself:
$$\lim_{T\to\infty}{|\tilde V_T(\omega)|^2\over 2T}$$
where the limit is taken to get rid of the truncation, it turns out that such a quantity (known as the periodogram has weak convergence properties. However, an ensemble average turns out to be convergent again and provides a good or at least well-behaved definition for the power spectrum:
$$S(\omega)=\lim_{T\to\infty}\left\langle{|\tilde V_T(\omega)|^2\over 2T}\right\rangle\,.$$
The main contribution from Wiener was to prove that the Fourier transform of the autocorrelation function is equal to this mathematically sound definition of the power spectrum
$$\lim_{T\to\infty}\left\langle{|\tilde V_T(\omega)|^2\over 2T}\right\rangle=\int_{-\infty}^\infty\langle V(t)^*V(t+\tau)\rangle e^{i\omega\tau}\,dt$$
Thereby, from the previous statements, providing the following definition for the power spectrum, namely, that initially suggested (without proof) by Einstein:
$$S(\omega)=\int_{-\infty}^\infty\langle V(t)^*V(t+\tau)\rangle e^{i\omega\tau}\,dt$$
The contribution of Khinchin was to extend the domain of validity to stochastic signals, as Wiener restrained his consideration to deterministic ones.
References
- I talked about the topic at the Mathematics & Computer Science Research Seminar of the University of Wolverhampton on 28 April (2022): Why we don't need the Wiener-Khinchin theorem.