Puzzle 14

Bulletin Editor Anirban DasGupta sets this problem. Student members of the IMS are invited to submit solutions (to bulletin@imstat.org with subject “Student Puzzle Corner”). The deadline is June 7, 2016.

It is the turn of a statistics problem this time, and we are going to look at a Bayesian answer to a historically celebrated problem. Give or take half a billion years, the estimated age of our Sun is 4.5 billion years. We think that on each day during these 4.5 billion years, the Sun has risen. Can we talk about the probability that it will rise everyday for the next 4.5 billion years? This is essentially what’s known as Laplace’s Sunrise problem.

A simple Bayes formulation that lets us do some calculations is as follows. Take the sequence of sunrises to be iid Bernoullis $X_1, X_2, \cdots $ with a success probability $p$, and give $p$ a prior distribution $G$ with a density $g$. Now calculate the Bayesian predictive probability $P(X_{n+1} = X_{n+2} = \cdots = X_{2n} = 1\,|X_1 = X_2 = \cdots = X_n = 1)$, induced by the joint marginal distribution of $(X_1, X_2, \cdots , X_{2n})$ under the prior distribution $G$.

Here is the exact problem of this issue:

For $n \geq 1$, define $\theta_n = P(X_{n+1} = X_{n+2} = \cdots = X_{2n} = 1\,|X_1 = X_2 = \cdots = X_n = 1)$.

(a) Take the special case where $g(p)$ is a Beta$(\alpha , \beta )$ density and find in closed form $\lim_{n \to \infty }\theta _n$.

(b) Take a general prior density $g$ and assume that $g$ is infinitely
differentiable from the left at $p = 1$. Find in closed form $\lim_{n \to \infty }\theta _n$.

Note: Part (b) is for extra credit and a personal letter from the Editor. The answer will depend on
the first $k$ for which $g^{(k)}(1-) \neq 0$.

Solution to Puzzle 13

Editor Anirban DasGupta writes:
The problem asked was the following: consider iid observations $X_1, X_2, \cdots $ distributed as $N(\mu , \sigma ^2)$ and consider for each $n, X_{(n)} = \max\{X_1, \cdots , X_n\}$ and $\bar{X} = \bar{X}_n = \frac{\sum_{i = 1}^nX_i}{n}$. Look at the conditional expectation of $X_{(n)}$ given $\bar{X}$, $\mu_n(\bar{X}) = E(X_{(n)}\,|\bar{X})$ and the conditional variance of $X_{(n)}$ given $\bar{X}$,
$V_n(\bar{X}) = \mbox{Var}(X_{(n)}\,|\bar{X})$. Find closed form deterministic (i.e., nonrandom) sequences $a_n, b_n, c_n, d_n$ such that $b_n\,[\mu_n(\bar{X})-a_n]$ and $d_n\,[V_n(\bar{X})-c_n]$ converge almost surely to one.

The intuition is that sample extremes and sample means are asymptotically independent, and so, asymptotically,
$\mu_n(\bar{X})$ should grow like the unconditional expectation of $X_{(n)}$ and $V_n(\bar{X})$ should decay like the unconditional variance of $X_{(n)}$.

However, no almost sure conclusions can be drawn from asymptotic independence or weak convergence. One must rigorously prove the result expected from informal intuition}.

It is enough to consider the standard normal case, namely, the case $\mu = 0, \sigma = 1$. In the standard normal case, by using Basu’s theorem, \[ \mu_n(\bar{X}) = E(X_{(n)}\,|\bar{X}) = \bar{X} + E(X_{(n)}-\bar{X}\,|\bar{X}) \] \[ = \bar{X} + E(X_{(n)}-\bar{X}) = \bar{X} + E(X_{(n)}). \]
Using the fact that $\frac{E(X_{(n)})}{\sqrt{2\log n}} \to 1$, and the Kolmogorov strong law, $\frac{\mu_n(\bar{X})}{\sqrt{2\log n}} \stackrel {a.s.} {\rightarrow } 1$.

Likewise, once again by using Basu’s theorem, in the standard normal case, \[ V_n(\bar{X}) = \mbox{Var}(X_{(n)}\,|\bar{X}) = \mbox{Var}(X_{(n)} – \bar{X}) \] \[ = \mbox{Var}(X_{(n)}) + \mbox{Var}(\bar{X}) -2\mbox{Cov}(X_{(n)},\bar{X}) = \mbox{Var}(X_{(n)}) + \mbox{Var}(\bar{X})-2\mbox{Var}(\bar{X}) = \mbox{Var}(X_{(n)})-\frac{1}{n}. \]

Use now uniform integrability of the sequence $X_{(n)}$, centered and normalized as \[ \sqrt{2\log n}\,[X_{(n)} – \sqrt{2\log n} + \frac{\log \log n + \log 4\pi }{2\sqrt{2\log n}}], \] and its weak convergence to the standard Gumbel law which has a variance of $\frac{\pi ^2}{6}$ to conclude that $\frac{\mbox{Var}(X_{(n)})}{\frac{\pi ^2}{12\,\log n}} \to 1$.

Thus, $\frac{12\,\log n}{\pi ^2}\,V_n(\bar{X}) \stackrel {a.s.} {\rightarrow } 1$.