View source for LBan-1



= Overview =
This lesson is structured in three parts:

* '''Self-averaging and disorder in statistical systems'''

Disordered systems are characterized by a random energy landscape, however, in the thermodynamic limit, physical observables become deterministic. This property, known as '''self-averaging''', does not always hold for the partition function which is the quantity that we can compute. When it holds the '''annealed average''' <math> \ln \overline{Z} </math> and the quenched average <math>  \overline{\ln Z}  </math> coincides otherwiese we have

<center> <math>   \overline{\ln Z} < \ln \overline{Z} </math> </center>
* '''The Random Energy Model'''

We study the Random Energy Model (REM) introduced by Bernard Derrida. In this model at each configuration is assigned an independent energy drawn from a Gaussian distribution of extensive variance. The model exhibits a '''freezing transition''' at a critical temperature, below which the free energy becomes dominated by the lowest energy states. 
* '''Extreme value statistics and saddle-point analysis'''

The results obtained from a saddle-point approximation can be recovered using the tools of extreme value statistics.

=Part I=
== Random energy landascape ==
In a system with <math>N</math> degrees of freedom, the number of configurations grows exponentially with <math>N</math>. For simplicity, consider Ising spins that take two values, <math>\sigma_i = \pm 1</math>, located on a lattice of size <math>L</math> in <math>d</math> dimensions. In this case, <math>N = L^d</math> and the number of configurations is <math>M = 2^N = e^{N \log 2}</math>.

In the presence of disorder, the energy associated with a given configuration becomes a random quantity. For instance, in the Edwards-Anderson model:
<center><math> E = - \sum_{\langle i, j \rangle} J_{ij} \sigma_i \sigma_j, </math></center>

where the sum runs over nearest neighbors <math>\langle i, j \rangle</math>, and the couplings <math>J_{ij}</math> are independent and identically distributed (i.i.d.) Gaussian random variables with zero mean and unit variance.

The energy of a given configuration is a random quantity because each system corresponds to a different realization of the disorder. In an experiment, this means that each of us has a different physical sample; in a numerical simulation, it means that each of us has generated a different set of couplings <math>J_{ij}</math>.


To illustrate this, consider a single configuration, for example the one where all spins are up. The energy of this configuration is given by the sum of all the couplings between neighboring spins:
<center><math> E[\sigma_1=1,\sigma_2=1,\ldots] = - \sum_{\langle i, j \rangle} J_{ij}. </math></center> Since the the couplings are random, the energy associated with this particular configuration is itself a Gaussian random variable, with zero mean and a variance proportional to the number of terms in the sum — that is, of order <math>N</math>.  The same reasoning applies to each of the <math>M = 2^N</math> configurations. So, in a disordered system, the entire energy landscape is random and sample-dependent.


== Self-averaging observables ==

A crucial question is whether the macroscopic properties measured on a given sample are themselves random or not. Our everyday experience suggests that they are not: materials like glass, ceramics, or bronze have well-defined, reproducible physical properties that can be reliably controlled for industrial applications.

From a more mathematical point of view, it means that the free energy  <math> F_N(\beta)=N f_N(\beta)</math> and its derivatives (magnetization, specific heat, susceptibility, etc.), in the limit <math> N \to \infty </math>, these random quantities concentrates around a well defined value. These observables are called self-averaging. This means that,
<center> 
<math>
\lim_{N \to \infty} f_N (\beta)= \lim_{N \to \infty}  f_N^{\text{typ}}(\beta) =\lim_{N \to \infty}  \overline{f_N(\beta)} =f_\infty(\beta)
</math>
</center>
Hence <math> f_N(\beta) </math> becomes effectively deterministic and its sample-to sample fluctuations vanish in relative terms:
<center> 
<math>
\lim_{N \to \infty} \frac{\overline{f_N^2(\beta)}}{\overline{f_N(\beta)}^2}=1.
</math>
</center>

==The partition function==
The partition function 

<center> <math> Z_N = \exp(-\beta N f_N(\beta)) </math></center>
is itself a random variable in disordered systems. Analytical methods can capture the statistical properties of this variable. We can define to average over the disorder realizations:
* The '''annealed average''' corresponds to the calculation of the moments of the partition function. The annealed free energy is 
<center> <math> f^{\text{ann.}} = -\frac{1}{\beta N} \ln \, \overline{Z_N} </math> </center>
*the '''quenched average''' corresponds to the average of the logarithm of the partition function, which is self-averaging for sure.  
<center><math> f_\infty (\beta)\sim \overline{f_N(\beta)}  = - \overline{ \ln Z_N(\beta)}/ (\beta N)</math></center>


Do these two averages coincide?

If the partition function is self-averaging in the thermodynamic limit, then
<center> 
<math>
\lim_{N \to \infty} Z_N(\beta) = \lim_{N \to \infty}  Z_N^{\text{typ}}(\beta) =\lim_{N \to \infty}  \overline{Z_N(\beta)} = e^{-\beta N f_\infty(\beta)}
</math>
</center>
As a consequence, the annealed and the quenched averages coincide. 

If the partition function is not self-averaging, only typical partition function concentrates, but extremely rare configurations contribute disproportionately to its moments:
<center> 
<math>
 \lim_{N \to \infty}  Z_N^{\text{typ}}(\beta) = e^{-\beta N f_\infty(\beta)}  < \lim_{N \to \infty}  \overline{Z_N(\beta)} = e^{-\beta N f^{\text{ann.}}(\beta)}
</math>
</center>


There are then two main strategies to determine the deterministic value of the observable :

* Compute directly the quenched average <math> f_\infty (\beta ) = - \overline{ \ln Z_N(\beta)}/ (\beta N)</math>  using methods such as the replica trick and the Parisi solution.

* Determine the typical value <math> Z_N^{\text{typ}}(\beta) </math> and evaluate   <math> f_\infty(\beta) = -\frac{1}{\beta N} \ln Z_N^{\text{typ}}(\beta) </math>

=Part II=
==Random Energy Model==
The Random energy model (REM) neglects the correlations between the <math>M=2^N</math> configurations. The energy associated to each configuration is an independent Gaussian variable with zero mean and variance <math>N</math>. The simplest solution of the model is with the microcanonical ensemble.


===Microcanonical calculation===

'''Step 1: Number of states '''. 

Let <math>{\mathcal N}_N(E) d E</math> the number of states of energy in the interval (E,E+dE).  
It is a random number and we use the representation 
<center ><math> \mathcal{N}_N(E)dE \equiv \exp(S_N(E))= \sum_{\alpha=1}^{2^N} \chi_\alpha(E) dE \;</math> </center>  
with <math> \chi_\alpha(E)=1</math> if  <math> E_\alpha \in [E, E+dE]</math> and  <math> \chi_\alpha(E)=0</math> otherwise.
We can cumpute its average
<center><math>  \overline{\mathcal{N}_N(E)} = \sum_{\alpha=1}^{2^N} \overline{\chi_\alpha(E)} = \frac{2^N}{\sqrt{2 \pi N}} \exp\left(-\frac{E^2}{2 N}\right) \sim \exp \left[N (\ln 2 -\epsilon^2/2)\right] </math></center>
Here <math> \epsilon =E/N </math> is the energy density and the  annealed  entropy density in the thermodynamic limit is
<center><math> s^{\text{ann.}}(\epsilon)=\ln 2 -\epsilon^2/2 </math></center>
'''Step 2: Self-averaging'''. 

Let compute now the second moment
<center ><math> \overline{\mathcal{N}_N^2(E)} = \sum_{\alpha=1}^{2^N} \overline{\chi_\alpha} \left(\sum_{\beta\ne \alpha} \overline{\chi_\beta} \right) + \sum_{\alpha=1}^{2^N} \overline{\chi_\alpha^2} \sim \overline{ \mathcal{N}_N(E)} \left( \overline{\mathcal{N}_N(E)} - \exp\left(-\frac{E^2}{2 N}\right) \right) + \overline{\mathcal{N}_N(E)} </math> </center>
We can then check the self averaging condition:
<center ><math> \frac{\overline{\mathcal{N}_N^2(E)}}{\overline{\mathcal{N}_N(E)}^2} \sim 1+ \frac{1}{\overline{\mathcal{N}_N(E)}}</math> </center>
A critical energy density <math>\epsilon^* = \sqrt{2 \ln 2}</math> separates a  '''self-averaging regime''' for <math>|\epsilon| < \epsilon^*</math> and a '''non self-averaging regime''' where for <math>|\epsilon| > \epsilon^*</math>.  In the first regime, <math>\overline{\mathcal{N}_N(E)}</math> is exponentially large and its value is determinstic (average, typical, median are the same). In the secon regime, <math>\overline{\mathcal{N}_N(E)}</math> is exponentially small but nonzero. The typical value instead is exactly zero, <math>\mathcal{N}_N^{\text{typ}}(E) = 0</math>: for most disorder realizations, there are no configurations with energy below <math>- \epsilon^* N</math> and only a vanishingly small fraction of rare samples gives a positive contribution to the average. As a result, the quenched average on the entropy density is: 
<center> <math> s_\infty(\epsilon) = \begin{cases} \ln 2 - \dfrac{\epsilon^2}{2}, & \text{for } |\epsilon| < \epsilon^* \\ -\infty, & \text{for } |\epsilon| > \epsilon^* \end{cases} </math> </center>

===Back to canonical ensemble: the freezing transition===
 
The '''annealed partition function''' is the average of the partition function over the disorder:
<center> <math> \overline{Z_N(\beta)} = \int_{-\infty}^{\infty} d\epsilon \; \overline{\mathcal{N}_N(\epsilon)} \, e^{-\beta N \epsilon} =   \int_{-\infty}^{\infty} d\epsilon \;  \exp \left[N (\ln 2 -\epsilon^2/2 -\beta \epsilon)\right]. </math> </center>
Using the saddle point for large N we find <math> \epsilon_{saddle} =-\beta</math> and thus
<center> <math> f^{\text{ann.}}(\beta) = - \ln 2/\beta - \dfrac{\beta}{2}</math> </center>

The '''quenched partition function''' is obtained replacing the mean with the typical value: 
<center> <math> Z_N^{\text{typ.}}(\beta) = \int_{-\infty}^{\infty} d\epsilon \; \mathcal{N}_N^{\text{typ.}}(\epsilon) \, e^{-\beta N \epsilon} =   \int_{-\epsilon^*}^{\epsilon^*} d\epsilon \;  \exp \left[N (\ln 2 -\epsilon^2/2 -\beta \epsilon)\right]. </math> </center>
Using the saddle point for large N we find a critical inverse temperature <math>\beta_c = \epsilon^* = \sqrt{2 \ln 2}</math> separating two phases:
* For <math>\beta < \beta_c </math>, <math> \epsilon_{saddle} =-\beta</math> and the annealed calculation works
*For <math>\beta > \beta_c </math>, <math> \epsilon_{saddle} =-\beta_c</math> and the free energy freezes to a temperature independent value. As a result, the quenched average on the free energy density is: 
<center> <math> f_\infty(\beta) = \begin{cases} -\ln 2/\beta - \dfrac{\beta}{2}, & \text{for } \beta < \beta_c \\ -\sqrt{2 \ln 2}, & \text{for } \beta> \beta_c \end{cases} </math> </center>

=Part III=
==Detour: Extreme Value Statistics==

Consider the REM spectrum of  <math>M</math> energies <math>E_1, \dots, E_M</math> drawn from a distribution <math>p(E)</math>. It is useful to introduce the cumulative probability of finding an energy smaller than ''E''
<center> <math>P(E) = \int_{-\infty}^E dx \, p(x)</math> </center>
We also define:
<center> <math>E_{\min} = \min(E_1, \dots, E_M), \quad Q_M(E) \equiv \text{Prob}(E_{\min} > E) </math> </center>

The statistical properties of <math>E_{\min}</math> are derived using two key relations:
*    '''First relation''':  
<center> <math>P(E_{\min}^{\text{typ}}) = 1/M</math> </center> 
This is an estimation of the typical value of the minimum. It is a crucial relation that will be used frequently.
*  '''Second relation''':
    <center> <math>Q_M(E) = (1-P(E))^M= e^{M \log(1 - P(E))} \sim \exp\left(-M P(E)\right) </math> </center> 
The first two steps are exact, but the resulting distribution depends on <math>M</math> and the precise form of <math>p(E)</math>. In contrast, the last step is an approximation, valid for large <math>M</math> and that  allows one to express the random variable <math>E_{\min}</math> in a scaling form: <math>E_{\min} = a_M + b_M z</math>, where <math>a_M</math> and <math>b_M</math> are deterministic and <math>M</math>-dependent, while <math>z</math> is a random variable that is independent of <math>M</math>.

== Gaussian Case ==

We ask you to prove that for a Gaussian distribution with zero mean and variance <math>\sigma^2</math>, the cumulative can be written as:

<center><math> P(E) = \exp(A(E)), \quad \text{with  } A(E)= -\frac{E^2}{2 \sigma^2} - \log\!\!\left(\frac{\sqrt{2 \pi}\, |E|}{\sigma}\right)+\ldots, \quad \text{and } A'(E)= -\frac{E}{\sigma^2} +\ldots</math></center>

* '''Typical Minimum''': From the first relation <math>A(E_{\min}^{\text{typ}}) = - \ln M</math> one obtains, for large \(M\):

<center><math>E_{\min}^{\text{typ}} = -\sigma \sqrt{2 \log M} \left( 1 - \frac{1}{4} \frac{\log (4 \pi \log M)}{\log M} + \ldots \right)</math></center>

* '''Gumbel Scaling''':  From the second relation, we look for a value <math>a_M</math>  such that <math>A(a_M) = - \ln M </math>, and then expand <math>A(E)</math> around <math>a_M</math>:

<center><math>Q_M(E) \sim \exp\!\!\Bigl(-M P(E)\Bigr) \sim \exp\!\!\left[-M \exp\!\!\bigl(A(a_M) + A'(a_M)(E-a_M)\bigr)\right]</math></center>

Hence, setting

<center><math>b_M = \frac{1}{A'(a_M)} = \frac{\sigma}{\sqrt{2 \log M}}</math></center>

the random variable

<center><math>z = \frac{E_{\min} - a_M}{b_M}</math></center>

is <math>M</math>-independent and Gumbel distributed:

<center><math>\pi(z) = \exp(z)\,\exp(-e^{z})</math></center>
== Back to REM ==

In the REM, the variance of the energies scales with the system size as
<center><math>\sigma_M^2 = \frac{\log M}{\log 2} = N.</math></center>
As a consequence, the minimum energy takes the form

<center><math>
E_{\min} = a_M + b_M z
= - \sqrt{2 \log 2}\, N
+ \frac{1}{2}\, \frac{\log (4 \pi N \log 2)}{\sqrt{2 \log 2}}
+ \frac{z}{\sqrt{2 \log 2}},
</math></center>

where <math>z</math> is a Gumbel-distributed random variable.

'''Key observations:'''
* At zero temperature (<math>\beta=\infty</math>), the ground-state energy is self-averaging. Its leading contribution is deterministic and extensive,  <math>F_N(\beta=\infty)\sim -\sqrt{2 \log 2}\, N</math>.

* Sample-to-sample fluctuations are subextensive (''N''-independent), with a finite standard deviation  <math>\sigma = \sqrt{2 \log 2}</math>.   We will later see that this scale coincides with the critical inverse temperature of the model.