Ya’acov Ritov is professor in the Department of Statistics at the Hebrew University of Jerusalem. He received his PhD from the Hebrew University in 1983, and is a fellow of the IMS. Ya’acov’s (statistical) research interests include complex and large dimensional model, empirical Bayes procedures, semi- and non-parametric models. His Medallion Lecture is also at JSM, on Thursday August 8, at 8:30am (see below for the times and locations of other Medallion Lectures, as well as the Wald Lectures, the Rietz Lecture and the Presidential Address.)
A Priori Analysis of Complex Models
We (P.J. Bickel, A.C. Gamst, B.J.K. Kleijn, and Y. Ritov) study a few examples of Bayesian procedures on complex, high-dimensional parameter spaces. The Bayesian procedures we consider are those that adhere to the following paradigm. The prior distribution is announced prior to observing the data. At least we are restricted to priors that do not depend on details of the experimental design or on knowing the specific functions of the parameters that may turn out to be of interest. In this paradigm, it would not, for example, be reasonable for a statistician to use one prior for estimating one function, and another to estimate a different function. We shouldn’t be reminded of Groucho Marx’s quote, “Those are my principles, and if you don’t like them… well, I have others.”
Bayesian procedures can be considered from different points of view. Their closure is the set of admissible procedures, and they are known to generate asymptotic minimax procedures in regular parametric models. However, these claims are valid when the priors are selected to fit frequentist ad-hoc considerations.
Most early discussions of Bayesian analysis presented simple examples, e.g., X ~ N(ϑ, 1). In this case, a statistician might have clear a priori ideas about ϑ, and might well understand the implications of using his prior. Regardless, the data will eventually overwhelm the prior, and typically frequentist and Bayesian inference will coincide. The classical Bernstein–von Mises Theorem encapsulates this observation. Currently, Bayesian procedures are being applied to complex, high-dimensional models, e.g., those used in medical imaging. With a very high-dimensional parameter space (where, for example, laws of large numbers appear in surprising places), it is very difficult to understand the implications of using a particular prior. It is very difficult, if not impossible, to express subjective information about the model in a robust prior, and it is difficult to express this knowledge in a way that would support the data analysis and not dominate it.
We use several examples to illustrate a number of issues. This includes the partial linear model of Engle, Granger, Rice and Weiss (1986) , and different models in the very convenient lab of white noise series. We show that in situations where the nonparametric part of the model is smooth enough, the Bernstein–von Mises phenomenon holds and Bayesian estimators are efficient, but the Bayesian estimator is going to fail in extreme situations where simple frequentist estimation can still work. Then, it may argue that in a given white noise model, the any Bayesian prior would fail in estimation of some linear functional, while trivial frequentist estimator would not.
We also give an example in which Bayesian procedures which ignore the stopping time associated with the data generating process fail, while simple frequentist procedures continue to work. This demonstrates the danger of the classical principle that Bayesians need not pay attention to stopping times.