T-5

Goal: So far we have discussed the equilibrium properties of disordered systems, that are encoded in their partition function and free energy. In this set of problems, we characterize the energy landscape of the spherical Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle p} -spin, by determining the number of its stationary points.

Key concepts: gradient descent, out-of-equilibrium dynamics, metastable states, Hessian matrices, random matrix theory, Langevin dynamics,?

Dynamics, optimization, trapping local minima

• Energy landscapes. Consider the spherical Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle p} -spin model discussed in the Problems 2 and 3; The function Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle E(\vec{\sigma})} is an energy landscape: it is a random function defined on configuration space, which is the space all configurations Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \vec{\sigma}} belong to. This landscape has its global minimum(a) at the ground state configuration(s): the energy density of the ground state(s) can be obtained studying the partition function Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle Z } in the limit Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \beta \to \infty } . Besides the ground state(s), the energy landscape can have other local minima; the fully-connected models of glasses are characterized by the fact that there are plenty of these local minima, see SKETCH.


• Gradient descent and stationary points. Suppose that we are interested in finding the configurations of minimal energy of some model with energy landscape $E({\vec {\sigma }})$, starting from an arbitrary initial configuration Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \vec{\sigma}_0} : we can think about a dynamics in which we progressively update the configuration of the system moving towards lower and lower values of the energy, hoping to eventually converge to the ground state(s). The simplest dynamics of this sort is gradient descent, Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \frac{d \vec{\sigma}(t)}{dt}=- \nabla_{\perp} E(\vec{\sigma}) }

  where the configuration changes in time moving in the direction of the gradient of the energy landscape restricted to the sphere, Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \nabla_{\perp} E(\vec{\sigma})} . The dynamics stops when it reaches a stationary point , i.e. a configuration where Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \nabla_\perp E(\vec{\sigma})=0} . If the landscape has a simple, convex structure, this will be the ground state one is seeking for; if the energy landscape is very non-convex like in glasses, the end point of this algorithm will be a local minimum at energies much higher than the ground state. SKETCH



• The landscape’s complexity. To understand the structure of the energy landscape and to guess where gradient descent dynamics (or its variation) are expected to converge, it is useful to characterize the distribution of the stationary points, i.e. the number Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathcal{N}(\epsilon)} of such configuration having a given energy density Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \epsilon } . In fully-connected models of glasses, this quantity has an exponential scaling, Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathcal{N}(\epsilon) \sim \text{exp}\left(N \Sigma(\epsilon) \right)} , where Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \Sigma(\epsilon)} is the complexity of the landscape. ^[1]

Problem 5.1: the Kac-Rice method and the complexity

In this Problem, we set up the computation of the annealed complexity of the spherical $p$-spin model, which is defined by

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \Sigma_{\text{a}}(\epsilon)= \lim_{N \to \infty}\frac{1}{N}\log \overline{\mathcal{N}(\epsilon)} , \quad \quad \mathcal{N}(\epsilon)= \left\{ \text{number stat. points of energy density } \epsilon\right\} }

1. The Kac-Rice formula. Consider first a random function of one variable Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle f(x)} defined on an interval $[a,b]$, and let Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathcal{N}} be the number of points Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle x } such that $f(x)=0$. Justify why

   Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \overline{\mathcal{N}}= \int_a^b dx \, \overline{\delta(f(x)) |f'(x)|} }

   In particular, why is the derivative of the function appearing in this formula? Consider now the number of stationary points Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathcal{N}(\epsilon)} of the $p$-spin energy landscape, which satisfy Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \nabla_\perp E(\vec{\sigma})=0} . Justify why the generalization of the formula above gives

   Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \overline{\mathcal{N}(\epsilon)}= \int_{S_N} d \vec{\sigma} \,\; \overline{|\text{det} \nabla_\perp^2 E (\vec{\sigma})|\,\, \delta(\nabla_\perp E(\vec{\sigma})=0) \, \,\delta(E(\vec{\sigma})- N \epsilon)} }

   where $\nabla _{\perp }^{2}E({\vec {\sigma }})$ is the Hessian matrix of the function Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle E (\vec{\sigma}) } restricted to the sphere.^[2]

2. Statistical rotational invariance. Recall the expression of the correlations of the energy landscape of the Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle p} -spin computed in Problem 2.1: in which sense the correlation function is rotationally invariant? Justify why rotational invariance implies that

   ${\overline {{\mathcal {N}}(\epsilon )}}=(2\pi e)^{\frac {N}{2}}\,\;{\overline {|{\text{det}}\nabla _{\perp }^{2}E({\vec {1}})|\,\,\delta (\nabla _{\perp }E({\vec {1}})=0)\,\,\delta (E({\vec {1}})-N\epsilon )}}$

   where Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \vec{1}=(1,1,1, \cdots, 1) } . Where does the prefactor arise from?

3. Gaussianity and correlations. Determine the distribution of the quantity Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle E (\vec{1})} . Show that the components of the vector $\nabla E({\vec {1}})$ are also Gaussian random variables with zero mean and covariances

   Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \overline{(\nabla E)_i \, (\nabla E)_j}= \frac{N}{2} p \, \delta_{ij} }

   The quantity Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \nabla_\perp E (\vec{1})} can be shown to be uncorrelated to $E({\vec {1}}),\nabla _{\perp }^{2}E({\vec {1}})$. Moreover, in the notation of ^[2] , Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \nabla^2_\perp E(\vec{1})= \hat \Pi(\vec{1}) \, \nabla^2 E(\vec{1}) \, \hat\Pi(\vec{1}) - p E(\vec{1}) \mathbb{I}} .

   Using this, show that

   Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \overline{\mathcal{N}(\epsilon)}= (2 \pi e)^{\frac{N}{2}} \,\frac{1}{(\pi N p)^{\frac{N}{2}}}\; \sqrt{\frac{N}{\pi}}e^{-N \epsilon^2}\; \overline{|\text{det} \left(\hat \Pi \nabla^2 E (\vec{1}) \hat \Pi - p N \epsilon \mathbb{I} \right)|}. }

   It remains to compute the expectation value of the determinant: this is the goal of the next problem.

Problem 5.2: the Hessian and random matrix theory

AGGIUSTA NORMALIZZAZIONE N-1 In this problem, we determine the average of the determinant of the Hessian matrix and conclude the calculation of the annealed complexity. The entries of the $(N-1)\times (N-1)$ matrix Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \nabla_\perp^2 E (\vec{\sigma}) } are also Gaussian variables. Computing their correlation, one finds that the matrix can be written as

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle [\nabla_\perp^2 E(\vec{1})]_{\alpha \beta}= \left[\hat \Pi \nabla^2 E (\vec{1}) \hat \Pi - p N \epsilon \mathbb{I} \right]_{\alpha \beta}= N \left( G_{\alpha \beta}- p \epsilon\, \delta_{\alpha \beta} \right), }

where the matrix $G$ has random entries with zero average and correlations

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \overline{{G}_{\alpha \beta} \, {G}_{\gamma \delta}}= \frac{p (p-1)}{2 N} \left( \delta_{\alpha \gamma} \delta_{\beta \delta}+ \delta_{\alpha \delta} \delta_{\beta \gamma}\right) }

1. . Show that the matrix Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle G } is a GOE matrix, i.e. a matrix taken from the Gaussian Orthogonal Ensemble, meaning that its distribution is

   Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P(G)= \frac{1}{Z_N} e^{-\frac{N}{4 \sigma^2} \text{Tr} G^2}, \quad \quad \sigma^2=\frac{p (p-1)}{2} }

   What’s is the value of Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \sigma^2 } ?

2. Let Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \lambda_\alpha } be the eigenvalues of the matrix $G$. Show that the following identity holds:

   Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \overline{|\text{det} \left(\hat \Pi \nabla^2 E (\vec{1}) \hat \Pi - p N \epsilon \mathbb{I} \right)|}= N^{N-1} \overline{\text{exp} \left[(N-1) \left( \int d \lambda \, \rho_N(\lambda) \, \log |\lambda - p \epsilon|\right) \right]}, \quad \quad \rho_{N}(\lambda)= \frac{1}{N-1} \sum_{\alpha=1}^{N-1} \delta (\lambda- \lambda_\alpha) }

   where Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \rho_{N}(\lambda)} is the empirical eigenvalue density.

3. . Concentration: the empirical density has a distribution of the large deviation form (see TD1) with speed DEFINE SPEED Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle N^2 } , meaning that Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P[\rho] = e^{-N^2 \, g[\rho]} } where now Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle g } is a functional (a function of a function). Using a saddle point argument, show that this implies

   Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \overline{\text{exp} \left[N \left( \int d \lambda \, \rho_N(\lambda) \, \log |\lambda - p \epsilon|\right) \right]}=\text{exp} \left[N \left( \int d \lambda \, \rho_{\text{ty}}(\lambda) \, \log |\lambda - p \epsilon|\right)+ o(N) \right] }

   where Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \rho_{\text{ty}}(\lambda) } is the typical value of the eigenvalue density. This quantity is self averaging, and for a GOE equals to

   Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \rho_N = \overline{\rho_N}= \rho_{\text{ty}}(\lambda)= \sqrt{\lambda^2 } }

   - check numerically - show that the resulting complexity is - plot this quantity, and determine numerically where it vanishes. Why the corresponding energy density must coincide with (average energy for beta to infty)

Notes

[1] - This quantity looks similar to the entropy Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle S(\epsilon) } we computed for the REM in Problem 1.1. However, while the entropy counts all configurations at a given energy density, the complexity Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \Sigma(\epsilon) } accounts only for the stationary points.

[2] - We define with Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \hat \Pi(\vec{\sigma}) } the projector on the tangent plane to the sphere at Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \vec{\sigma}} : this is the plane orthogonal to the vector Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \vec{\sigma}} . The gradient Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \nabla_\perp E(\vec{\sigma}) } is a Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle (N-1)} -dimensional vector that is obtained projecting the gradient Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle [\nabla E(\vec{\sigma})]_i=\partial E/\partial \sigma_i } on the tangent plane, Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \nabla_\perp E(\vec{\sigma})=\hat \Pi \nabla E(\vec{\sigma})} . The Hessian Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \nabla^2_\perp E(\vec{\sigma}) } is a Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle (N-1) \times (N-1)} -dimensional matrix that is obtained from the Hessian $[\nabla ^{2}E({\vec {\sigma }})]_{ij}=\partial ^{2}E/\partial \sigma _{i}\partial \sigma _{j}$ as Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \nabla^2_\perp E(\vec{\sigma})= \hat \Pi(\vec{\sigma}) \, \nabla^2 E(\vec{\sigma}) \, \hat\Pi(\vec{\sigma}) - \nabla E(\vec{\sigma}) \cdot \vec{\sigma} \mathbb{I}} where Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbb{I}} is the identity matrix.