Piet Groeneboom has been professor of statistics at Delft University since 1988, having previously been professor of statistics at the University of Amsterdam. He earned his PhD in Mathematics in 1979 under the direction of J. Oosterhoff. He has been visiting professor at the University of Washington, Seattle, Stanford University and Université Paris VI and has done research in the areas of large deviations, stochastic geometry, particle systems, inverse statistical problems and statistical inference under order restrictions. Piet Groeneboom has been on the editorial board of the Annals of Statistics (three times) and is IMS fellow, elected member of the ISI, and a recipient of the Rollo Davidson Prize. Presently he is finishing a book to be published by Cambridge University Press, with co-author Geurt Jongbloed, on the topic of his Wald lectures. Piet will present his Wald Lectures at the JSM in Montreal on Tuesday, August 6, and Wednesday, August 7.

Nonparametric estimation under shape constraints

Research on nonparametric estimation under shape constraints started in the fifties of the preceding century, with papers by (among others) Daniel Brunk and Constance van Eeden on estimation of parameters under the restriction of monotonicity or unimodality. Ronald Pyke states in his discussion on a paper of Daniel Brunk at a conference in Bloomington, Indiana, in 1970: “The adjective ‘isotonic’ is a relatively new term which has been introduced by the author of this paper. To several people in the Northwest corners of the United States, this concept has for a number of years been commonly referred to as Brunkizing.”. An isotonic estimator is an estimator which is computed under an order restriction, where the order can be a partial order. The order restriction can also be put on the derivative of the estimator, and in this sense the estimator of a convex function (in dimension 1 or higher), which is itself also convex, is also an isotonic estimator.

A summary of the early work was given in the well-known book of the “four B’s” (Barlow, Bartholomew, Bremner and Brunk) on isotonic regression. This book appeared in 1972, but is (unfortunately) out of print now. Originally, the focus was on defining the estimators satisfying these order constraints. As an example, the maximum likelihood estimator of a monotone decreasing density was in 1957 proved by Grenander to be the left-continuous slope of the least concave majorant of the empirical distribution function. Developing distribution theory for these estimators turned out to be rather difficult and is now commonly classified as belonging to the area of “non-standard asymptotics”, with non-normal limit distributions and rates of convergence slower than the square root of the sample size.

In a pioneering paper, published in 1964, Herman Chernoff showed that a nonparametric estimator of the mode of a unimodal distribution has a limit distribution which can be characterized by the solution of a heat equation under certain boundary conditions, and in 1969 Prakasa Rao showed that Grenander’s maximum likelihood estimator of a smooth strictly decreasing density also has this limit distribution. If the density is strictly decreasing and smooth the local limit distribution is the distribution of the location of the maximum of Brownian motion minus a parabola.

The original computations of this limit distribution were based on numerical solutions of Chernoff ’s heat equation. However, by the rather awkward boundary conditions, one cannot expect that numerical solutions, obtained in this way, will be very accurate. But around 1984, several authors discovered (independently) that the limit distribution has an analytic representation in terms of Airy functions. This means that nowadays one can compute the limit density quickly and accurately, using computer algebra packages such as Mathematica.

Simulations of the limiting distribution of the estimator were found not to be very accurate and one might wonder whether some form of bootstrapping will work. A negative result in this direction was proved in 2008 by Kosorok, who showed that bootstrapping from the empirical distribution function will produce an inconsistent estimate of the local limit distribution of Grenander’s estimate of a decreasing density. In continuation of this, Sen, Banerjee and Woodroofe have a result (2010) which suggests that bootstrapping from the Grenander estimate itself will also not reproduce the local limit behavior (the proof is presently still not complete, though).

The estimators under restriction of monotonicity have a rather different limit behavior if the underlying density or regression function is not strictly monotone. For example, if the underlying density is uniform, Grenander’s density estimate does not have as local limit distribution the distribution of the location of the maximum of Brownian motion minus a parabola. In that case the local limit can be described by the least concave majorant of the Brownian bridge and the limit behavior of global distances can be characterized using properties of the least concave majorant of Brownian motion without drift. In fact, it has been proved that the vertices of the least concave majorant of Brownian motion are generated by an inhomogeneous Poisson process of which the Poisson measure can be explicitly characterized. This topic has recently been taken up again by Jim Pitman, Fadoua Balabdaoui and others who have extended and generalized the results which were obtained in this area in the eighties of the preceding century.

Research on isotonic regression got a new impetus in the nineties when it became clear that it was the right setting for studying estimators of the distribution function in inverse problems, such as current status or more generally interval censoring models. Here we see again the limit distribution of Grenander’s density estimator appear, but this time not as the limit distribution of a density estimator, but as the limit distribution of the estimator of a distribution function, which is monotone by definition. Whether this really gives the limit behavior of the nonparametric maximum likelihood estimator of the distribution function for interval censoring is still an open question though, although a very specific conjecture for the convergence to this limit distribution has been formulated. For the case where the interval to which the variable of interest belongs is not arbitrarily small (the so-called “strict separation case”) the limit distribution has been determined. And for the simplest case of interval censoring, current status, this limit distribution had already been derived in 1987. In the current status model the only information one has about the data of interest is provided by an observation time and the information telling us whether the event of interest happened before the observation time or still did still not happen. Current status data or interval censored data are quite common in medical research, but are also relevant for econometric models like the binary choice model.

Other functionals of Brownian motion appear in the limit theory for nonparametric estimators of convex functions. In particular, the local limit of a nonparametric least squares estimator of a smooth convex regression function can be characterized as the second derivative of the “invelope” of integrated Brownian motion motion plus the 4th power of the time variable, instead of Brownian motion with parabolic drift, which figured in the limit distribution of monotone estimators. Also, the rate of convergence is faster in the convex case than in the monotone case.

The theory is equally relevant for isotonic nonparametric estimators in deconvolution problems, where most of the limit theory is still unknown. Further development of the theory will depend on the ability to deal with certain integral equations, associated with these models. These integral equations were first studied in the 90’s for the case of interval censoring, and were used to show that certain smooth functionals of the model (like moments) could be estimated efficiently, using the MLE, at the usual $\sqrt {n}$ rate with normal limit distributions. Progress in this field is rather slow, though, and there remain lots of open problems.