T-5

Goal: So far we have discussed the equilibrium properties of disordered systems, that are encoded in their partition function and free energy. In this set of problems, we characterize the energy landscape of a prototypical model, the spherical Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle p} -spin.

Key concepts: Langevin dynamics, gradient descent, oout-of-equilibrium dynamics, metastable states, Hessian matrices, random matrix theory.

Dynamics, optimization, trapping local minima

• Energy landscapes. Consider the spherical Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle p} -spin model; The function Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle E(\vec{\sigma})} defines the energy landscape of the model: this is a random function defined on configuration space, which is the space all configurations ${\vec {\sigma }}$ belong to. This landscape has its global minima in the ground state configurations: the energy density of the ground states can be obtained studying the partition function Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle Z } in the limit Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \beta \to \infty } . Besides the ground state(s), the energy landscape can have other local minima; the models of glasses are characterize by the fact that there are plenty of these local minima, see the sketch.


• Gradient descent and stationary points. Suppose that we are interested in finding the configurations of minimal energy of some model with energy landscape $E({\vec {\sigma }})$, starting from an arbitrary initial configuration Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \vec{\sigma}_0} : we can think about a dynamics in which we progressively update the configuration of the system moving towards lower and lower values of the energy, hoping to eventually converge to the ground state(s). The simplest dynamics of this sort is gradient descent, where the configurations change in time moving in the direction of the gradient of the energy landscape: Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \frac{d \vec{\sigma}(t)}{dt}=- \nabla_{\perp} E(\vec{\sigma}) }

  Under this dynamics, the system descends in the energy landscape towards configurations of lower and lower energy, until it reaches a stationary point, i.e. a configuration where $\nabla E({\vec {\sigma }})=0$: at that point, the dynamics stops. If the energy landscape has a simple, convex structure, the stationary point will be the ground state one is seeking for; however, if the energy landscape is very non-convex like in glasses, the end point of this algorithm will likely be a local minimum at energies much higher than the ground state. SKETCH



• Noise, Langevin dynamics and activation. How can one modify the dynamics to escape from a given local minimum and explore other regions of the energy landscape? One possibility is to add some stochasticity (or noise), i.e. some random terms that kick the systems in random directions in configuration space, towards which maybe the energy increases instead of decreasing: Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \frac{d \vec{\sigma}(t)}{dt}=- \nabla E(\vec{\sigma})+ \vec{\eta}(t) }

  The simplest choice is to choose Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \vec{\eta}(t)} to be a Gaussian vector at each time $t$, uncorrelated from the vectors at other times Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle t' \neq t } , with zero average and some constant variance. This variance, which measures the strength of the noisy kicks, can be interpreted as a temperature: the resulting dynamics is known as Langevin dynamics .

Problem 5.1: the Kac-Rice method and the complexity

1. The Kac-Rice formula I. Consider first a function of one variable Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle f(x)} defined on an interval $[a,b]$, and let Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathcal{N}} be the number of points Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle x } such that $f(x)=0$. One has

   Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathcal{N}= \int_a^b dx \, \delta(f(x)) |f'(x)| }

   Why is the derivative off the function appearing in this formula? Justify why if Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle f} is a random function, the average of this number can be written as

   ${\overline {\mathcal {N}}}=\int _{a}^{b}dx\,\mathbb {E} [|f'(x)|{\Big |}f(x)=0]\,\mathbb {P} _{x}(f(x)=0)$

   where Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbb{P}_{x}(f(x)=0)} is the probability density of Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle f(x)} computed at zero, while $\mathbb {E} [\cdot {\Big |}E]$ is the expectation value of a random variable conditioned to the fact that an event Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle E } is true.

2. The Kac-Rice formula II. Consider now the number of stationary points Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathcal{N}(\epsilon)} of the $p$-spin energy landscape, which satisfy Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \nabla_\perp E(\vec{\sigma})} . Justify why the generalization of the formula above gives

   Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \overline{\mathcal{N}(\epsilon)}= \int_{S_N} d \vec{\sigma} \, \mathbb{E}[ |\text{det} \nabla_\perp^2 E (\vec{\sigma})| \Big| \nabla_\perp E=0, E= N \epsilon] \, \mathbb{P}_{x}(\nabla_\perp E=0, E= N \epsilon) }

   We now use this formula to compute the annealed complexity Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \Sigma_{\text{a}}(\epsilon) } , which is defined by ${\overline {{\mathcal {N}}(\epsilon )}}\sim e^{N\Sigma _{\text{a}}(\epsilon )+o(N)}$. We do the calculation in three step.

3. Step 1: use rotational invariance. Recall the expression of the correlations of the energy landscape of the Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle p} -spin computed in Problem 2.1: in which sense the correlation function is rotationally invariant?

3. Step 2: a problem of Gaussian variables.

4. Step 3: random determinants and eigenvalue density.