Groupe 01 13: /* Preliminaries: the central limit */

2011-11-29T11:15:46Z

Preliminaries: the central limit

← Older revision		Revision as of 13:15, 29 November 2011
Line 34:		Line 34:
	* compute the mean value and the variance of this distribution		* compute the mean value and the variance of this distribution
	* consider <math>X_N</math>, the sum of ''N'' independent, exponentially distributed, random variables. How <math>X_N</math> is distributed?		* consider <math>X_N</math>, the sum of ''N'' independent, exponentially distributed, random variables. How <math>X_N</math> is distributed?
	* You can check your result with this Python program

	We write <math>X_N</math> in a more convenient way		We write <math>X_N</math> in a more convenient way

Lptmswiki: Created page with "FORCETOC = Extreme Statistics = Generically, finding the distribution of the maximum of a set of random variables is a non-trivial problem, which appears in many contex..."

2011-09-05T09:54:44Z

Created page with "__FORCETOC__ = Extreme Statistics = Generically, finding the distribution of the maximum of a set of random variables is a non-trivial problem, which appears in many contex..."

New page

__FORCETOC__

= Extreme Statistics =

Generically, finding the distribution of the maximum of a set of random variables is a non-trivial problem, which appears in many contexts ranging from the maximal height of water in a river to fluctuations in stock markets
We consider ''N'' independent random variables <math>(x_1,...,x_N)</math> drawn from the same distribution <math>p(x)</math>.
We denote
<center><math>y_N=\max(x_1,...,x_N)</math></center>

It is useful to use the following notations for the cumulative distributions
<center><math>P^<(x)=\int_{-\infty}^x dx' p(x')\qquad\qquad P^>(x)=\int_x^{+\infty} dx' p(x') </math></center>

Let us denote by <math>q_N(y)</math> the distribution of <math>y_N</math> and by <math>Q_N(y)=\text{Prob}(y_N<y)</math> its cumulative distribution.

* Write <math>Q_N(y)</math> in terms of <math>P^<(y) </math>. (Help: Start to write this relation for <math>N=2,3,...</math>).
This is the fundamental relation of Extreme statistics and we analyze its consequences in the large ''N'' limit where, analogously to the central limit theorem, extremes statistics display universal features.
* In particular shows that in the large ''N'' limit we can write
<center><math>Q_N(y) \sim \exp\left(-N P^>(y)\right) </math></center>

In the present exercise, we first study the case of the exponential distribution. In a second step we generalize our results to a larger class of distributions.

==Exponential distribution==

The exponential distribution is one of the fundamental continuous distributions, and already for this reason worthy of study. Among many other places, it appears in the Poisson process. The distribution writes:

<center><math>p(x) = \lambda \exp(-\lambda x)</math></center>
where both <math> \lambda</math> and <math>x</math> are positive numbers.
===Preliminaries: the central limit===

* compute the mean value and the variance of this distribution
* consider <math>X_N</math>, the sum of ''N'' independent, exponentially distributed, random variables. How <math>X_N</math> is distributed?
* You can check your result with this Python program

We write <math>X_N</math> in a more convenient way
<center><math>X_N = a_N + b_N z </math></center>
where <math>a_N </math> the location of the distribution and <math>b_N </math> is the width of the distribution of <math>X_N </math>. Both numbers depend on <math>N </math>. Finally, <math>z </math> is a random number and its distribution, <math>\pi(z)</math> becomes independent of <math>N </math> in the large "N" limit. In other words this means that the distribution of <math>X_N</math> is significantly different from zero when the value of <math>X_N </math> is around <math>a_N </math>, in a region of size <math>b_N </math>.

* From the central limit theorem which is the natural choice for <math>a_N </math> and <math>b_N </math>? Write the distribution <math>\pi(z)</math>

===The Maxima===
Consider now the case <math>\lambda=1</math>
* Write <math>P^>(x)</math> and <math>P^<(x)</math>. (Remember that <math>x</math> is a positive number.)
* Write <math>Q_N(y)</math> and <math>q_N(y)</math>.
* Plot <math>q_N(y)</math> for different values of ''N''.

We want now to give a natural definition for the number <math>a_N</math> and <math>b_N</math>.

Consider <math>P^>(\tilde y)=\frac 12</math>. If you draw N independent exponential variables, how many variables (in average) will be greater than <math>\tilde y</math>? Repeat the same exercise with <math>\tilde \tilde y</math> such that
<math>P^>( \tilde \tilde y)=\frac 23</math>

* Justify that <math>a_N</math> can be estimated from
<center><math>P^>(a_N)=\frac 1N</math></center>
* Compute <math>a_N</math> for the exponential distribution and justify that
<center><math>Q_N\left(y=a_N+z\right)</math></center>

In the large ''N'' limit, the distribution <math>\pi(z)</math> becomes <math>N</math> independent.

* Show that in this limit its cumulative takes the from
<center><math>\Pi(z)= e^{-e^{-z}}</math></center>
This is the cumulative distribution of the famous Gumbel distribution.

Let us remark that the precise definition of <math>a_N</math> and <math>b_N</math> fix the mean and the variance of the rescaled distribution <math>\pi(z)</math>
At variance with the central limit case the mean will be different from zero and the variance different from one.
* Compute the mean, the variance and the asymptotic behavior of the Gumbel distribution. Draw the distribution. Explain why <math>z=0</math> is a special point

== Generic case: Universality of the Gumbel distribution ==

The Gumbel distribution is the limit distribution of the maxima of a large class of function. We can say that the Gumbel distribution plays, for extreme statistics, the same role of the Gaussian distribution for the central limit theorem.

By contrast the behavior of <math>a_N</math> and <math>b_N</math> as a function of <math>N</math> strongly depend on the particular distributions <math>p(x)</math>. We discuss here a family of distribution characterized by a fast decay for large <math>x</math>
<center><math>p(x) \sim c e^{- x^\alpha}</math></center>
where <math>\alpha>0</math>
The key point is to be able to determine <math>A(x)</math> such that
<center><math>P^>(x)=\exp(-A(x))</math></center>
* For <math>p(x) = e^{- x}</math> shows <math>A(x)=x</math>
Otherwise <math>A(x)</math> should be determined asymptotically for large <math>x</math>
* Show that <math>A(x)=x^\alpha +(\alpha-1) \log x+...</math>
* Show that in general <math>A(a_N)= \log N+...</math> and compute <math>a_N</math> as a function of <math> \alpha </math> for large <math> N </math>.
* Show that the maximum distribution take the form
<center><math> \lim _{N\to \infty } Q_N(y)=\left( y= a_N+ \frac{z}{A'(a_N)} \right)</math></center>
with <math> z </math> Gumbel distributed
* Identify <math>b_N</math> and discuss its behavior as a function of <math> \alpha </math>

If the distribution <math>p(x)</math> is defined on the entire real axis and is characterized by the same fast decay, it is easy to generalize this result also for the distribution of the minima.

*Write the Gumbel distribution for the minima

==Minimum of exponential random numbers==
The Gumbel distribution is not the only distribution for the extremes. Consider the simple case of the minima of the exponential distribution
* Show analytically that the distribution function for the minimum of <math>N</math> exponential random numbers <math>x = \min(x_1, \dots, x_N) </math> with parameters <math>\lambda_1, \dots \lambda_N</math> is again an exponential random number with parameter <math>\lambda_1 + \cdots + \lambda_N</math>:<br>
<math> \pi(x)= (\lambda_1 + \cdots + \lambda_N) \exp(-(\lambda_1 + \cdots + \lambda_N)x)</math><br> Program this in Python, produce a histogram and compare with the exact result.
* Look on the web which are the possible extreme distributions for independent and identically distributed variable

T-II-3 - Revision history

Groupe 01 13: /* Preliminaries: the central limit */

Lptmswiki: Created page with "__FORCETOC__ = Extreme Statistics = Generically, finding the distribution of the maximum of a set of random variables is a non-trivial problem, which appears in many contex..."

Lptmswiki: Created page with "FORCETOC = Extreme Statistics = Generically, finding the distribution of the maximum of a set of random variables is a non-trivial problem, which appears in many contex..."