LBan-1

Overview

This lesson is structured in three parts:

• Self-averaging and disorder in statistical systems

Disordered systems are characterized by a random energy landscape, however, in the thermodynamic limit, physical observables become deterministic. This property, known as self-averaging, does not always hold for the partition function which is the quantity that we can compute. When it holds the annealed average $\ln \bar{Z}$ and the quenched average $\overline{\ln Z}$ coincides otherwiese we have

$\overline{\ln Z}<\ln \bar{Z}$

• The Random Energy Model

We study the Random Energy Model (REM) introduced by Bernard Derrida. In this model at each configuration is assigned an independent energy drawn from a Gaussian distribution of extensive variance. The model exhibits a freezing transition at a critical temperature​, below which the free energy becomes dominated by the lowest energy states.

• Extreme value statistics and saddle-point analysis

The results obtained from a saddle-point approximation can be recovered using the tools of extreme value statistics.

Part I

Random energy landascape

In a system with $N$ degrees of freedom, the number of configurations grows exponentially with $N$. For simplicity, consider Ising spins that take two values, ${\sigma }_i=\pm 1$, located on a lattice of size $L$ in $d$ dimensions. In this case, $N=L^d$ and the number of configurations is $M=2^N=e^{N\log 2}$.

In the presence of disorder, the energy associated with a given configuration becomes a random quantity. For instance, in the Edwards-Anderson model:

$E=-\sum _{⟨i,j⟩}{J}_{ij}{\sigma }_i{\sigma }_j,$

where the sum runs over nearest neighbors $⟨i,j⟩$, and the couplings $J_{ij}$ are independent and identically distributed (i.i.d.) Gaussian random variables with zero mean and unit variance.

The energy of a given configuration is a random quantity because each system corresponds to a different realization of the disorder. In an experiment, this means that each of us has a different physical sample; in a numerical simulation, it means that each of us has generated a different set of couplings $J_{ij}$.

To illustrate this, consider a single configuration, for example the one where all spins are up. The energy of this configuration is given by the sum of all the couplings between neighboring spins:

$E[{\sigma }_1=1,{\sigma }_2=1,\dots ]=-\sum _{⟨i,j⟩}{J}_{ij}.$

Since the the couplings are random, the energy associated with this particular configuration is itself a Gaussian random variable, with zero mean and a variance proportional to the number of terms in the sum — that is, of order

$N$

. The same reasoning applies to each of the

$M=2^N$

configurations. So, in a disordered system, the entire energy landscape is random and sample-dependent.

Self-averaging observables

A crucial question is whether the macroscopic properties measured on a given sample are themselves random or not. Our everyday experience suggests that they are not: materials like glass, ceramics, or bronze have well-defined, reproducible physical properties that can be reliably controlled for industrial applications.

From a more mathematical point of view, it means that the free energy $F_N(\beta )=N{f}_N(\beta )$ and its derivatives (magnetization, specific heat, susceptibility, etc.), in the limit $N\to \mathrm{\infty }$, these random quantities concentrates around a well defined value. These observables are called self-averaging. This means that,

$\underset{N\to \mathrm{\infty }}{\lim }{f}_N(\beta )=\underset{N\to \mathrm{\infty }}{\lim }{f}_N^{\text{typ}}(\beta )=\underset{N\to \mathrm{\infty }}{\lim }\overline{f_N(\beta )}=f_{\mathrm{\infty }}(\beta )$

Hence $f_N(\beta )$ becomes effectively deterministic and its sample-to sample fluctuations vanish in relative terms:

$\underset{N\to \mathrm{\infty }}{\lim }\frac{\overline{f_N^2(\beta )}}{\stackrel{2}{\stackrel{‾}{f_N(\beta )}}}=1.$

The partition function

The partition function

$Z_N=\exp (-\beta N{f}_N(\beta ))$

is itself a random variable in disordered systems. Analytical methods can capture the statistical properties of this variable. We can define to average over the disorder realizations:

• The annealed average corresponds to the calculation of the moments of the partition function. The annealed free energy is

$f^{\text{ann.}}=-\frac{1}{\beta N}\ln \overline{Z_N}$

• the quenched average corresponds to the average of the logarithm of the partition function, which is self-averaging for sure.

$f_{\mathrm{\infty }}(\beta )\sim \overline{f_N(\beta )}=-\overline{\ln Z_N(\beta )}/(\beta N)$

Do these two averages coincide?

If the partition function is self-averaging in the thermodynamic limit, then

$\underset{N\to \mathrm{\infty }}{\lim }{Z}_N(\beta )=\underset{N\to \mathrm{\infty }}{\lim }{Z}_N^{\text{typ}}(\beta )=\underset{N\to \mathrm{\infty }}{\lim }\overline{Z_N(\beta )}=e^{-\beta N{f}_{\mathrm{\infty }}(\beta )}$

As a consequence, the annealed and the quenched averages coincide.

If the partition function is not self-averaging, only typical partition function concentrates, but extremely rare configurations contribute disproportionately to its moments:

$\underset{N\to \mathrm{\infty }}{\lim }{Z}_N^{\text{typ}}(\beta )=e^{-\beta N{f}_{\mathrm{\infty }}(\beta )}<\underset{N\to \mathrm{\infty }}{\lim }\overline{Z_N(\beta )}=e^{-\beta N{f}^{\text{ann.}}(\beta )}$

There are then two main strategies to determine the deterministic value of the observable :

• Compute directly the quenched average $f_{\mathrm{\infty }}(\beta )=-\overline{\ln Z_N(\beta )}/(\beta N)$ using methods such as the replica trick and the Parisi solution.

• Determine the typical value $Z_N^{\text{typ}}(\beta )$ and evaluate $f_{\mathrm{\infty }}(\beta )=-\frac{1}{\beta N}\ln Z_N^{\text{typ}}(\beta )$

Part II

Random Energy Model

The Random energy model (REM) neglects the correlations between the $M=2^N$ configurations. The energy associated to each configuration is an independent Gaussian variable with zero mean and variance $N$. The simplest solution of the model is with the microcanonical ensemble.

Microcanonical calculation

Step 1: Number of states .

Let ${{𝒩}}_N(E)dE$ the number of states of energy in the interval (E,E+dE). It is a random number and we use the representation

${{𝒩}}_N(E)dE\equiv \exp (S_N(E))=\sum _{\alpha =1}^{2^N}{\chi }_{\alpha }(E)dE$

with ${\chi }_{\alpha }(E)=1$ if $E_{\alpha }\in [E,E+dE]$ and ${\chi }_{\alpha }(E)=0$ otherwise. We can cumpute its average

$\overline{{{𝒩}}_N(E)}=\sum _{\alpha =1}^{2^N}\overline{{\chi }_{\alpha }(E)}=\frac{2^N}{\sqrt{2\pi N}}\exp (-\frac{E^2}{2N})\sim \exp [N(\ln 2-{ϵ}^2/2)]$

Here $ϵ=E/N$ is the energy density and the annealed entropy density in the thermodynamic limit is

$s^{\text{ann.}}(ϵ)=\ln 2-{ϵ}^2/2$

Step 2: Self-averaging.

Let compute now the second moment

$\overline{{{𝒩}}_N^2(E)}=\sum _{\alpha =1}^{2^N}\overline{{\chi }_{\alpha }}\left(\sum _{\beta \ne \alpha }\overline{{\chi }_{\beta }}\right)+\sum _{\alpha =1}^{2^N}\overline{{\chi }_{\alpha }^2}\sim \overline{{{𝒩}}_N(E)}(\overline{{{𝒩}}_N(E)}-\exp (-\frac{E^2}{2N}))+\overline{{{𝒩}}_N(E)}$

We can then check the self averaging condition:

$\frac{\overline{{{𝒩}}_N^2(E)}}{\stackrel{2}{\overline{{{𝒩}}_N(E)}}}\sim 1+\frac{1}{\overline{{{𝒩}}_N(E)}}$

A critical energy density ${ϵ}^{*}=\sqrt{2\ln 2}$ separates a self-averaging regime for $|ϵ|<{ϵ}^{*}$ and a non self-averaging regime where for $|ϵ|>{ϵ}^{*}$. In the first regime, $\overline{{{𝒩}}_N(E)}$ is exponentially large and its value is determinstic (average, typical, median are the same). In the secon regime, $\overline{{{𝒩}}_N(E)}$ is exponentially small but nonzero. The typical value instead is exactly zero, ${{𝒩}}_N^{\text{typ}}(E)=0$: for most disorder realizations, there are no configurations with energy below $-{ϵ}^{*}N$ and only a vanishingly small fraction of rare samples gives a positive contribution to the average. As a result, the quenched average on the entropy density is:

$s_{\mathrm{\infty }}(ϵ)=\{\begin{array}{ll}\ln 2-\frac{{ϵ}^2}{2},& \text{for }|ϵ|<{ϵ}^{*}\\ -\mathrm{\infty },& \text{for }|ϵ|>{ϵ}^{*}\end{array}$

Back to canonical ensemble: the freezing transition

The annealed partition function is the average of the partition function over the disorder:

$\overline{Z_N(\beta )}=\underset{-\mathrm{\infty }}{\overset{\mathrm{\infty }}{\int }}dϵ\overline{{{𝒩}}_N(ϵ)}{e}^{-\beta Nϵ}=\underset{-\mathrm{\infty }}{\overset{\mathrm{\infty }}{\int }}dϵ\exp [N(\ln 2-{ϵ}^2/2-\beta ϵ)].$

Using the saddle point for large N we find ${ϵ}_{saddle}=-\beta$ and thus

$f^{\text{ann.}}(\beta )=-\ln 2/\beta -\frac{\beta }{2}$

The quenched partition function is obtained replacing the mean with the typical value:

$Z_N^{\text{typ.}}(\beta )=\underset{-\mathrm{\infty }}{\overset{\mathrm{\infty }}{\int }}dϵ{{𝒩}}_N^{\text{typ.}}(ϵ)e^{-\beta Nϵ}=\underset{-{ϵ}^{*}}{\overset{{ϵ}^{*}}{\int }}dϵ\exp [N(\ln 2-{ϵ}^2/2-\beta ϵ)].$

Using the saddle point for large N we find a critical inverse temperature ${\beta }_c={ϵ}^{*}=\sqrt{2\ln 2}$ separating two phases:

• For $\beta <{\beta }_c$, ${ϵ}_{saddle}=-\beta$ and the annealed calculation works
• For $\beta >{\beta }_c$, ${ϵ}_{saddle}=-{\beta }_c$ and the free energy freezes to a temperature independent value. As a result, the quenched average on the free energy density is:

$f_{\mathrm{\infty }}(\beta )=\{\begin{array}{ll}-\ln 2/\beta -\frac{\beta }{2},& \text{for }\beta <{\beta }_c\\ -\sqrt{2\ln 2},& \text{for }\beta >{\beta }_c\end{array}$

Part III

Detour: Extreme Value Statistics

Consider the REM spectrum of $M$ energies $E_1,\dots ,E_M$ drawn from a distribution $p(E)$. It is useful to introduce the cumulative probability of finding an energy smaller than E

$P(E)=\underset{-\mathrm{\infty }}{\overset{E}{\int }}dxp(x)$

We also define:

$E_{\min }=\min (E_1,\dots ,E_M),Q_M(E)\equiv \text{Prob}(E_{\min }>E)$

The statistical properties of $E_{\min }$ are derived using two key relations:

• First relation:

$P(E_{\min }^{\text{typ}})=1/M$

This is an estimation of the typical value of the minimum. It is a crucial relation that will be used frequently.

• Second relation:

$Q_M(E)=(1-P(E){)}^M=e^{M\log (1-P(E))}\sim \exp (-MP(E))$

The first two steps are exact, but the resulting distribution depends on $M$ and the precise form of $p(E)$. In contrast, the last step is an approximation, valid for large $M$ and that allows one to express the random variable $E_{\min }$ in a scaling form: $E_{\min }=a_M+b_{M}z$, where $a_M$ and $b_M$ are deterministic and $M$-dependent, while $z$ is a random variable that is independent of $M$.

Gaussian Case

We ask you to prove that for a Gaussian distribution with zero mean and variance ${\sigma }^2$, the cumulative can be written as:

$P(E)=\exp (A(E)),\text{with }A(E)=-\frac{E^2}{2{\sigma }^2}-\log \left(\frac{\sqrt{2\pi }|E|}{\sigma }\right)+\dots ,\text{and }{A}^{\prime }(E)=-\frac{E}{{\sigma }^2}+\dots$

• Typical Minimum: From the first relation $A(E_{\min }^{\text{typ}})=-\ln M$ one obtains, for large \(M\):

$E_{\min }^{\text{typ}}=-\sigma \sqrt{2\log M}(1-\frac{1}{4}\frac{\log (4\pi \log M)}{\log M}+\dots )$

• Gumbel Scaling: From the second relation, we look for a value $a_M$ such that $A(a_M)=-\ln M$, and then expand $A(E)$ around $a_M$:

$Q_M(E)\sim \exp (-MP(E))\sim \exp [-M\exp (A(a_M)+A^{\prime }(a_M)(E-a_M)\left)\right]$

Hence, setting

$b_M=\frac{1}{A^{\prime }(a_M)}=\frac{\sigma }{\sqrt{2\log M}}$

the random variable

$z=\frac{E_{\min }-a_M}{b_M}$

is $M$-independent and Gumbel distributed:

$\pi (z)=\exp (z)\exp (-e^z)$

Back to REM

In the REM, the variance of the energies scales with the system size as

${\sigma }_M^2=\frac{\log M}{\log 2}=N.$

As a consequence, the minimum energy takes the form

$E_{\min }=a_M+b_{M}z=-\sqrt{2\log 2}N+\frac{1}{2}\frac{\log (4\pi N\log 2)}{\sqrt{2\log 2}}+\frac{z}{\sqrt{2\log 2}},$

where $z$ is a Gumbel-distributed random variable.

Key observations:

• At zero temperature ($\beta =\mathrm{\infty }$), the ground-state energy is self-averaging. Its leading contribution is deterministic and extensive, $F_N(\beta =\mathrm{\infty })\sim -\sqrt{2\log 2}N$.

• Sample-to-sample fluctuations are subextensive (N-independent), with a finite standard deviation $\sigma =\sqrt{2\log 2}$. We will later see that this scale coincides with the critical inverse temperature of the model.