LBan-1

Overview

This lesson is structured in three parts:

• Self-averaging and disorder in statistical systems

Disordered systems are characterized by a random energy landscape, however, in the thermodynamic limit, physical observables become deterministic. This property, known as self-averaging, does not always hold for the partition function which is the quantity that we can compute. When it holds the annealed average $\ln {\overline {Z}}$ and the quenched average ${\overline {\ln Z}}$ coincides otherwiese we have

${\overline {\ln Z}}<\ln {\overline {Z}}$

• The Random Energy Model

We study the Random Energy Model (REM) introduced by Bernard Derrida. In this model at each configuration is assigned an independent energy drawn from a Gaussian distribution of extensive variance. The model exhibits a freezing transition at a critical temperature​, below which the free energy becomes dominated by the lowest energy states.

• Extreme value statistics and saddle-point analysis

The results obtained from a saddle-point approximation can be recovered using the tools of extreme value statistics.

Part I

Random energy landascape

In a system with $N$ degrees of freedom, the number of configurations grows exponentially with $N$. For simplicity, consider Ising spins that take two values, $\sigma _{i}=\pm 1$, located on a lattice of size $L$ in $d$ dimensions. In this case, $N=L^{d}$ and the number of configurations is $M=2^{N}=e^{N\log 2}$.

In the presence of disorder, the energy associated with a given configuration becomes a random quantity. For instance, in the Edwards-Anderson model:

$E=-\sum _{\langle i,j\rangle }J_{ij}\sigma _{i}\sigma _{j},$

where the sum runs over nearest neighbors $\langle i,j\rangle$, and the couplings $J_{ij}$ are independent and identically distributed (i.i.d.) Gaussian random variables with zero mean and unit variance.

The energy of a given configuration is a random quantity because each system corresponds to a different realization of the disorder. In an experiment, this means that each of us has a different physical sample; in a numerical simulation, it means that each of us has generated a different set of couplings $J_{ij}$.

To illustrate this, consider a single configuration, for example the one where all spins are up. The energy of this configuration is given by the sum of all the couplings between neighboring spins:

$E[\sigma _{1}=1,\sigma _{2}=1,\ldots ]=-\sum _{\langle i,j\rangle }J_{ij}.$

Since the the couplings are random, the energy associated with this particular configuration is itself a Gaussian random variable, with zero mean and a variance proportional to the number of terms in the sum — that is, of order $N$. The same reasoning applies to each of the $M=2^{N}$ configurations. So, in a disordered system, the entire energy landscape is random and sample-dependent.

Self-averaging observables

A crucial question is whether the macroscopic properties measured on a given sample are themselves random or not. Our everyday experience suggests that they are not: materials like glass, ceramics, or bronze have well-defined, reproducible physical properties that can be reliably controlled for industrial applications.

From a more mathematical point of view, it means that the free energy $F_{N}(\beta )=Nf_{N}(\beta )$ and its derivatives (magnetization, specific heat, susceptibility, etc.), in the limit $N\to \infty$, these random quantities concentrates around a well defined value. These observables are called self-averaging. This means that,

$\lim _{N\to \infty }f_{N}(\beta )=\lim _{N\to \infty }f_{N}^{\text{typ}}(\beta )=\lim _{N\to \infty }{\overline {f_{N}(\beta )}}=f_{\infty }(\beta )$

Hence $f_{N}(\beta )$ becomes effectively deterministic and its sample-to sample fluctuations vanish in relative terms:

$\lim _{N\to \infty }{\frac {\overline {f_{N}^{2}(\beta )}}{{\overline {f_{N}(\beta )}}^{2}}}=1.$

The partition function

The partition function

$Z_{N}=\exp(-\beta Nf_{N}(\beta ))$

is itself a random variable in disordered systems. Analytical methods can capture the statistical properties of this variable. We can define to average over the disorder realizations:

• The annealed average corresponds to the calculation of the moments of the partition function. The annealed free energy is

$f^{\text{ann.}}=-{\frac {1}{\beta N}}\ln \,{\overline {Z_{N}}}$

• the quenched average corresponds to the average of the logarithm of the partition function, which is self-averaging for sure.

$f_{\infty }(\beta )\sim {\overline {f_{N}(\beta )}}=-{\overline {\ln Z_{N}(\beta )}}/(\beta N)$

Do these two averages coincide?

If the partition function is self-averaging in the thermodynamic limit, then

$\lim _{N\to \infty }Z_{N}(\beta )=\lim _{N\to \infty }Z_{N}^{\text{typ}}(\beta )=\lim _{N\to \infty }{\overline {Z_{N}(\beta )}}=e^{-\beta Nf_{\infty }(\beta )}$

As a consequence, the annealed and the quenched averages coincide.

If the partition function is not self-averaging, only typical partition function concentrates, but extremely rare configurations contribute disproportionately to its moments:

$\lim _{N\to \infty }Z_{N}^{\text{typ}}(\beta )=e^{-\beta Nf_{\infty }(\beta )}<\lim _{N\to \infty }{\overline {Z_{N}(\beta )}}=e^{-\beta Nf^{\text{ann.}}(\beta )}$

There are then two main strategies to determine the deterministic value of the observable :

• Compute directly the quenched average $f_{\infty }(\beta )=-{\overline {\ln Z_{N}(\beta )}}/(\beta N)$ using methods such as the replica trick and the Parisi solution.

• Determine the typical value $Z_{N}^{\text{typ}}(\beta )$ and evaluate Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle f_\infty(\beta) = -\frac{1}{\beta N} \ln Z_N^{\text{typ}}(\beta) }

Part II

Random Energy Model

The Random energy model (REM) neglects the correlations between the Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle M=2^N} configurations. The energy associated to each configuration is an independent Gaussian variable with zero mean and variance Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle N} . The simplest solution of the model is with the microcanonical ensemble.

Microcanonical calculation

Step 1: Number of states .

Let Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle {\mathcal N}_N(E) d E} the number of states of energy in the interval (E,E+dE). It is a random number and we use the representation

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathcal{N}_N(E)dE \equiv \exp(S_N(E))= \sum_{\alpha=1}^{2^N} \chi_\alpha(E) dE \;}

with Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \chi_\alpha(E)=1} if Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle E_\alpha \in [E, E+dE]} and Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \chi_\alpha(E)=0} otherwise. We can cumpute its average

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \overline{\mathcal{N}_N(E)} = \sum_{\alpha=1}^{2^N} \overline{\chi_\alpha(E)} = \frac{2^N}{\sqrt{2 \pi N}} \exp\left(-\frac{E^2}{2 N}\right) \sim \exp \left[N (\ln 2 -\epsilon^2/2)\right] }

Here Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \epsilon =E/N } is the energy density and the annealed entropy density in the thermodynamic limit is

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle s^{\text{ann.}}(\epsilon)=\ln 2 -\epsilon^2/2 }

Step 2: Self-averaging.

Let compute now the second moment

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \overline{\mathcal{N}_N^2(E)} = \sum_{\alpha=1}^{2^N} \overline{\chi_\alpha} \left(\sum_{\beta\ne \alpha} \overline{\chi_\beta} \right) + \sum_{\alpha=1}^{2^N} \overline{\chi_\alpha^2} \sim \overline{ \mathcal{N}_N(E)} \left( \overline{\mathcal{N}_N(E)} - \exp\left(-\frac{E^2}{2 N}\right) \right) + \overline{\mathcal{N}_N(E)} }

We can then check the self averaging condition:

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \frac{\overline{\mathcal{N}_N^2(E)}}{\overline{\mathcal{N}_N(E)}^2} \sim 1+ \frac{1}{\overline{\mathcal{N}_N(E)}}}

A critical energy density Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \epsilon^* = \sqrt{2 \ln 2}} separates a self-averaging regime for Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle |\epsilon| < \epsilon^*} and a non self-averaging regime where for Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle |\epsilon| > \epsilon^*} . In the first regime, Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \overline{\mathcal{N}_N(E)}} is exponentially large and its value is determinstic (average, typical, median are the same). In the secon regime, Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \overline{\mathcal{N}_N(E)}} is exponentially small but nonzero. The typical value instead is exactly zero, Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathcal{N}_N^{\text{typ}}(E) = 0} : for most disorder realizations, there are no configurations with energy below Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle - \epsilon^* N} and only a vanishingly small fraction of rare samples gives a positive contribution to the average. As a result, the quenched average on the entropy density is:

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle s_\infty(\epsilon) = \begin{cases} \ln 2 - \dfrac{\epsilon^2}{2}, & \text{for } |\epsilon| < \epsilon^* \\ -\infty, & \text{for } |\epsilon| > \epsilon^* \end{cases} }

Back to canonical ensemble: the freezing transition

The annealed partition function is the average of the partition function over the disorder:

${\overline {Z_{N}(\beta )}}=\int _{-\infty }^{\infty }d\epsilon \;{\overline {{\mathcal {N}}_{N}(\epsilon )}}\,e^{-\beta N\epsilon }=\int _{-\infty }^{\infty }d\epsilon \;\exp \left[N(\ln 2-\epsilon ^{2}/2-\beta \epsilon )\right].$

Using the saddle point for large N we find $\epsilon _{saddle}=-\beta$ and thus

$f^{\text{ann.}}(\beta )=\ln 2/\beta +{\dfrac {\beta }{2}}$

The quenched partition function is obtained replacing the mean with the typical value:

$Z_{N}^{\text{typ.}}(\beta )=\int _{-\infty }^{\infty }d\epsilon \;{\mathcal {N}}_{N}^{\text{typ.}}(\epsilon )\,e^{-\beta N\epsilon }=\int _{-\epsilon ^{*}}^{\epsilon ^{*}}d\epsilon \;\exp \left[N(\ln 2-\epsilon ^{2}/2-\beta \epsilon )\right].$

Using the saddle point for large N we find a critical inverse temperature $\beta _{c}=\epsilon ^{*}={\sqrt {2\ln 2}}$ separating two phases:

• For $\beta <\beta _{c}$, $\epsilon _{saddle}=-\beta$ and the annealed calculation works
• For $\beta >\beta _{c}$, $\epsilon _{saddle}=-\beta _{c}$ and the free energy freezes to a temperature independent value. As a result, the quenched average on the free energy density is:

$f_{\infty }(\beta )={\begin{cases}\ln 2/\beta +{\dfrac {\beta }{2}},&{\text{for }}\beta <\beta _{c}\\{\sqrt {2\ln 2}},&{\text{for }}\beta >\beta _{c}\end{cases}}$

Part III

Detour: Extreme Value Statistics

Consider the REM spectrum of $M$ energies $E_{1},\dots ,E_{M}$ drawn from a distribution $p(E)$. It is useful to introduce the cumulative probability of finding an energy smaller than E

$P(E)=\int _{-\infty }^{E}dx\,p(x)$

We also define:

$E_{\min }=\min(E_{1},\dots ,E_{M}),\quad Q_{M}(E)\equiv {\text{Prob}}(E_{\min }>E)$

The statistical properties of $E_{\min }$ are derived using two key relations:

• First relation:

$P(E_{\min }^{\text{typ}})=1/M$

This is an estimation of the typical value of the minimum. It is a crucial relation that will be used frequently.

• Second relation:

$Q_{M}(E)=(1-P(E))^{M}=e^{M\log(1-P(E))}\sim \exp \left(-MP(E)\right)$

The first two steps are exact, but the resulting distribution depends on $M$ and the precise form of $p(E)$. In contrast, the last step is an approximation, valid for large $M$ and that allows one to express the random variable $E_{\min }$ in a scaling form: $E_{\min }=a_{M}+b_{M}z$, where $a_{M}$ and $b_{M}$ are deterministic and $M$-dependent, while $z$ is a random variable that is independent of $M$.

Gaussian Case

We ask you to prove that for a Gaussian distribution with zero mean and variance Failed to parse (syntax error): {\displaystyle \(\sigma^2\)} , the cumulative can be written as:

$P(E)=\exp(A(E)),\quad {\text{with }}A(E)=-{\frac {E^{2}}{2\sigma ^{2}}}-\log \!\!\left({\frac {{\sqrt {2\pi }}\,|E|}{\sigma }}\right)+\ldots ,\quad {\text{and }}A'(E)=-{\frac {E}{\sigma ^{2}}}+\ldots$

• Typical Minimum:

From the first relation

$A(E_{\min }^{\text{typ}})=-\ln M$

one obtains, for large \(M\):

$E_{\min }^{\text{typ}}=-\sigma {\sqrt {2\log M}}\left(1-{\frac {1}{4}}{\frac {\log(4\pi \log M)}{\log M}}+\ldots \right)$

• Gumbel Scaling:

From the second relation, we look for a value 'a_M' such that \(A(a_M) = - \ln M\), and then expand \(A(E)\) around 'a_M':

$Q_{M}(E)\sim \exp \!\!{\Bigl (}-MP(E){\Bigr )}\sim \exp \!\!\left[-M\exp \!\!{\bigl (}A(a_{M})+A'(a_{M})(E-a_{M}){\bigr )}\right]$

Hence, setting

$b_{M}={\frac {1}{A'(a_{M})}}={\frac {\sigma }{\sqrt {2\log M}}}$

the random variable

$z={\frac {E_{\min }-a_{M}}{b_{M}}}$

is \(M\)-independent and Gumbel distributed:

$\pi (z)=\exp(z)\,\exp(-e^{z})$

Back to REM

In the REM the variance is $\sigma _{M}^{2}={\frac {\log M}{\log 2}}=N$. Then we have:

$E_{\min }=a_{M}+b_{M}z=-{\sqrt {2\log 2}}\;N+{\frac {1}{2}}{\frac {\log(4\pi \log 2N)}{\sqrt {2\log 2}}}+{\frac {z}{\sqrt {2\log 2}}}$

Key Observations:

• the ground state energy is self-averaging with an extensive deterministic part $\sim -{\sqrt {2\log 2}}\cdot N=f_{\infty }(\beta >\beta _{c})\cdot N$.
• Its fluctuations are very small (N independent) with a standard deviation ${\sqrt {2\log 2}}=\beta _{c}$.