Our Contributing Editor Yoram Gat responds to the article by Jeffrey Rosenthal in the previous issue.
Yoram writes:

The January/February 2021 issue of the Bulletin carried a column by Professor Jeffrey Rosenthal about the inaccurate predictions of pollsters regarding the outcomes of the November US presidential elections. Some statisticians saw such erroneous predictions as a reason for shame, an embarrassment for the profession, Prof. Rosenthal wrote, but he sees them as exciting because they “provide compelling ways to teach our students the importance of statistical assumptions, as well as new opportunities to investigate innovative ways to overcome their limitations”.

Prof. Rosenthal’s sanguine attitude toward the persistent phenomenon of erroneous polling predictions is in my opinion quite worrisome. We are not concerned here with theoretical exercises or with laboratory experiments. Polling, for better or worse, is part of public life and its performance has implications for society. Presumably no one would profess excitement at repeated failures of, say, the electricity grid, even if such failures can be used in the instruction of engineering students and can lead in the long term to useful improvements in the infrastructure.

To use a term that has gained some popularity in recent years, publicizing erroneous polling predictions must be thought of as the dissemination of fake news. Fake news, we are frequently told, is spread by extremists and by enemies without in order to undermine the trust citizens have in the institutions of our society. Disseminating misleading polling results does just that.

First of all, errors quite naturally reduce the credibility of those who have been proven wrong. The public really has little reason to put its faith in the reliability of those who repeatedly generate and publish wrong predictions. But more perilously, widely propagated false pronouncements by pollsters, like false pronouncements by experts in other areas of public life — economics, defense and public health come to mind — come to be perceived by the public (rightly or wrongly) as deliberate manipulation by the establishment.

Thus, errors in polling predictions are a matter for serious concern rather than for excitement. The response to errors should indeed be full recognition and thorough investigation, but instead of focusing on the hope that such investigation will lead at some point in the future to “overcoming the limitations” of existing statistical techniques, the immediate objective should be to make sure that published polling results clearly and appropriately reflect the limitations that exist. If and when new polling and statistical techniques are developed, purporting to produce more accurate results, those techniques should be tested rigorously before their results are offered to the public as being a reliable way to assess public opinion.

Public trust is a precious and scarce resource. Society without trust is a poor society. Statisticians and other experts who are given access to public attention should keep this in mind and act with the appropriate caution and humility.

Jeffrey Rosenthal responds:

Thanks for the feedback, Yoram. I certainly agree that published polling results should reflect their limitations. Perhaps they should include a warning similar to mutual fund statements that “Past performance is no guarantee of future results”. The public needs to understand that polls, like weather forecasts and Academy Award predictions, provide useful information but no certainties. Why? Because accurate polling is a very difficult challenge! And I see no need for statisticians to feel any shame or embarrassment about that.