The Wiener-Khinchin theorem
The Wiener-Khinchin theorem is one of the central results of statistical and stochastic physics and of signal processing. It states that the power spectrum $S(\omega)$ and the autocorrelation function $\langle\ud{a}(t+\tau)a(\tau)\rangle$ form a Fourier transform pair.
To appreciate the importance of this result, one needs to first understand what is a power spectrum, what is an autocorrelation function and why their connection as established by the theorem is, beyond useful for its practical purpose, insightful
The idea was first proposed by Einstein at a meeting of the Swiss "Société de Physique" whose compte-rendu lists the people who first heard about this connection (as well as what they discussed about themselves)
The communication was saved for posterity in a two-pages memo[1], written in French. Neither Einstein nor other—several prestigious—scientists seem to have appreciated or understood the significance of the statement at the time. Einstein himself discusses at length the technical feasibility of the integration, for which he consigns that his friend M. P. Habicht (not Conrad?) assured him:
M. P. Habicht, m'a montré en outre que la détermination des moyennes de (1) peuvent se faire aisément à l'aide d'un intégrateur mécanique de maniement facile.
One comment is recorded in the minutes of the meeting from the president Pierre Weiss who, despite his input to mean-field theory, also lacks a proper valuation of the result:
M. Weiss fait remarquer combien la méthode de M. Einstein rendra de services à la météorolog-ie très riche en matériaux qui, jusqu'à présent, étaient à peu près inutilisables.
Einstein drafted a longer version, that was, however, never published (it is available as Doc. 30 of the 4th volume of his collected papers (the two-page memo appears there as the previous Doc. 29).
The result comes in full bloom with Norbert Wiener, who, on opposite, writes almost a book (a 142 pages paper) on the topic of "Generalized harmonic analysis" [2].
A brief account of the main results is that energy typically comes as the square of the signal, i.e.,
$$E=\int_{-\infty}^\infty|V(t)|^2\,dt$$
for the total energy, while in the frequency space, from Plancherel's theorem, the same relation holds for the Fourier transform:
$$E=\int_{-\infty}^\infty|\tilde V(\omega)|^2\,dt$$
from which one can infer that $|\tilde V(\omega)|^2$ is the energy density, i.e., $|\tilde V(\omega)|^2\Delta\omega$ is how much energy the system has in the window $[\omega, \omega+\Delta\omega]$.
However, a stationary signal does not have, in general, a Fourier transform, but a truncated version $V_T$ does:
$$V_T(t)\equiv V(t)\mathbb{1}_{\{|t|\le T\}}$$
so that now one can define the FT $\tilde V(\omega)$:
$$ V_T(t)={1\over2\pi}\int_{-\infty}^\infty\tilde V_T(\omega)e^{i\omega t}\,d\omega$$
and while the following definition would match the initial association of a power spectrum to the signal itself:
$$\lim_{T\to\infty}{|\tilde V_T(\omega)|^2\over 2T}$$
where the limit is taken to get rid of the truncation, it turns out that such a quantity (known as the periodogram has weak convergence properties. However, an ensemble average turns out to be convergent again and provides a good or at least well-behaved definition for the power spectrum:
$$S(\omega)=\lim_{T\to\infty}\left\langle{|\tilde V_T(\omega)|^2\over 2T}\right\rangle\,.$$
The main contribution from Wiener was to prove that the Fourier transform of the autocorrelation function is equal to this mathematically sound definition of the power spectrum
$$\lim_{T\to\infty}\left\langle{|\tilde V_T(\omega)|^2\over 2T}\right\rangle=\int_{-\infty}^\infty\langle V(t)^*V(t+\tau)\rangle e^{i\omega\tau}\,dt$$
Thereby, from the previous statements, providing the following definition for the power spectrum, namely, that initially suggested (without proof) by Einstein:
$$S(\omega)=\int_{-\infty}^\infty\langle V(t)^*V(t+\tau)\rangle e^{i\omega\tau}\,dt$$
The contribution of Khinchin was to extend the domain of validity to stochastic signals, as Wiener restrained his consideration to deterministic ones.
References
- I talked about the topic at the Mathematics & Computer Science Research Seminar of the University of Wolverhampton on 28 April (2022): Why we don't need the Wiener-Khinchin theorem.