T-6

Goal: Complete the characterisation of the energy landscape of the spherical $p$-spin.
Techniques: saddle point, random matrix theory.

Problems

Problem 6: the Hessian and random matrix theory

To get the complexity, it remains to compute the expectation value of the determinant of the Hessian matrix: this is the goal of this problem. We will do this exploiting results from random matrix theory.

1. Gaussian Random matrices. Show that the matrix $M$ is a GOE matrix, i.e. a matrix taken from the Gaussian Orthogonal Ensemble, meaning that it is a symmetric matrix with distribution $P(M)=Z_{N}^{-1}e^{-{\frac {N}{4\sigma ^{2}}}{\text{Tr}}M^{2}}.$ What is the value of $\sigma ^{2}$?

2. Eigenvalue density and concentration. Let $\lambda _{\alpha }$ be the eigenvalues of the matrix $M$. Show that the following identity holds:

   ${\overline {|{\text{det}}\left(M-p\epsilon \mathbb {I} \right)|}}={\overline {{\text{exp}}\left[(N-1)\left(\int d\lambda \,\rho _{N}(\lambda )\,\log |\lambda -p\epsilon |\right)\right]}},\quad \quad \rho _{N}(\lambda )={\frac {1}{N-1}}\sum _{\alpha =1}^{N-1}\delta (\lambda -\lambda _{\alpha })$

   where $\rho _{N}(\lambda )$ is the empirical eigenvalue density. It can be shown that if $M$ is a GOE matrix, the distribution of the empirical density has a large deviation form (recall TD1) with speed $N^{2}$, meaning that $P_{N}[\rho ]=e^{-N^{2}\,g[\rho ]}$ where now $g[\cdot ]$ is a functional (a function of a function). Using a saddle point argument, show that this implies

   ${\overline {{\text{exp}}\left[(N-1)\left(\int d\lambda \,\rho _{N}(\lambda )\,\log |\lambda -p\epsilon |\right)\right]}}={\text{exp}}\left[N\left(\int d\lambda \,\rho _{\text{typ}}(\lambda +p\epsilon )\,\log |\lambda |\right)+o(N)\right]$

   where $\rho _{\text{typ}}(\lambda )$ is the typical value of the eigenvalue density, which satisfies $g[\rho _{\text{typ}}]=0$.

3. The semicircle and the complexity. The eigenvalue density of GOE matrices is self-averaging, and it equals to

   $\lim _{N\to \infty }\rho _{N}(\lambda )=\lim _{N\to \infty }{\overline {\rho _{N}}}(\lambda )=\rho _{\text{typ}}(\lambda )={\frac {1}{2\pi \sigma ^{2}}}{\sqrt {4\sigma ^{2}-\lambda ^{2}}}$

   • Check this numerically: generate matrices for various values of $N$, plot their empirical eigenvalue density and compare with the asymptotic curve. Is the convergence faster in the bulk, or in the edges of the eigenvalue density, where it vanishes?
   • Combining all the results, show that the annealed complexity is $\Sigma _{\text{a}}(\epsilon )={\frac {1}{2}}\log[4e(p-1)]-{\frac {\epsilon ^{2}}{2}}+I_{p}(\epsilon ),\quad \quad I_{p}(\epsilon )={\frac {2}{\pi }}\int dx{\sqrt {1-\left(x-{\frac {\epsilon }{\epsilon _{\text{th}}}}\right)^{2}}}\,\log |x|,\quad \quad \epsilon _{\text{th}}=-2{\sqrt {\frac {p-1}{p}}}.$

     The integral $I_{p}(\epsilon )$ can be computed explicitly, and one finds:

     $I_{p}(\epsilon )={\begin{cases}&{\frac {\epsilon ^{2}}{\epsilon _{\text{th}}^{2}}}-{\frac {1}{2}}-{\frac {\epsilon }{\epsilon _{\text{th}}}}{\sqrt {{\frac {\epsilon ^{2}}{\epsilon _{\text{th}}^{2}}}-1}}+\log \left({\frac {\epsilon }{\epsilon _{\text{th}}}}+{\sqrt {{\frac {\epsilon ^{2}}{\epsilon _{\text{th}}^{2}}}-1}}\right)-\log 2\quad {\text{if}}\quad \epsilon \leq \epsilon _{\text{th}}\\&{\frac {\epsilon ^{2}}{\epsilon _{\text{th}}^{2}}}-{\frac {1}{2}}-\log 2\quad {\text{if}}\quad \epsilon >\epsilon _{\text{th}}\end{cases}}$

     Plot the annealed complexity, and determine numerically where it vanishes: why is this a lower bound or the ground state energy density?

4. The threshold and the stability. Sketch $\rho _{\text{typ}}(\lambda +p\epsilon )$ for different values of $\epsilon$; recalling that the Hessian encodes for the stability of the stationary points, show that there is a transition in the stability of the stationary points at the critical value of the energy density $\epsilon _{\text{th}}=-2{\sqrt {(p-1)/p}}.$ When are the critical points stable local minima? When are they saddles? Why the stationary points at $\epsilon =\epsilon _{\text{th}}$ are called marginally stable ?

Check out: key concepts

Metastable states, Hessian matrices, random matrix theory, landscape’s complexity.