Contributing Editor Yoram Gat writes:

The practice of sortition—appointing decision-making bodies via statistical sampling—has been gaining ground in recent years. Very recently, an allotted body has been convened by the French government to propose ways to deal with climate change. In Belgium, a permanent advisory body appointed by sortition has been constituted. In 2016, allotted bodies wrote propositions for constitutional amendments that were then put up for popular referenda.

As allotted bodies come to wield significant political power, the issues associated with the way they are constituted gain importance and deserve critical scrutiny. Many of those issues are the same ones that affect the constitution of any decision-making body: determining the size of the body, the length of its term of service, its purview, its procedures, its budget, its professional staff, etc. Setting those constitutive parameters correctly would make the difference between having a body that is able to make policy based on the informed and considered ideas of its members, and one which is in fact merely a façade for established powers that use it to legitimate their own decisions.

However, allotted bodies do present a set of issues that are unique to their constitution: issues associated with the sampling procedure. Use of sortition is justified by the fact that it constitutes a body whose makeup reflects the makeup of the population affected by its decisions. It is therefore crucial that the sampling procedure is constructed in a way that enables a convincing argument to be made that the resulting sample is, for all practical purposes, an equal-probability sample from the entire affected population.

In recent applications of sortition, these issues have been dealt with in very problematic ways. The entire sampling procedure was usually undertaken out of sight of the public, sometimes handed off to a private contractor, with the public being expected to trust the outcome blindly. In addition, some of the details of the sampling procedure that were published clearly contradicted the requirement for equi-probability sampling.

In view of the deficiency of the de facto standards that are being established, I think that clear, science-based guidelines for the acceptability of a sortition procedure should be established. It seems that the expertise of statisticians in sampling procedures makes them (us) qualified for writing such guidelines.

Therefore, I would like to call upon the community of statisticians to produce, publish and promote a protocol, or a set of criteria, for properly designed sortition procedures. The protocol should allow the evaluation of the quality of proposed applications of sortition. The existence of such a protocol, if it is perceived as having been carefully and professionally done, can serve as a useful guide to those who are designing sortition procedure. It may be an important contribution to the democratization of the way decisions are made in modern society.

Here are some of the issues that a useful protocol for sortition would need to address:

1. Verifying that a reliable registry of the eligible allotment pool has been created.
What makes a such registry reliable? When allotting a national body, can the electoral rolls be used? In some countries (such as the U.S.) electoral rolls, or poll books, are an area for active political manipulation. If additional eligibility criteria need to be employed (e.g., being a resident of a particular area), this would require a manipulation-resistant procedure to establish eligibility according to those criteria.

2. Verifying that sampling from the registry is for all practical purposes equi-probable.
This may be far more difficult than it seems. Naturally, the randomization process should be public. However, randomization devices—mechanical or electronic—can be rigged by powerful adversaries (with the prize potentially being much more valuable than the prize in any money-prize lottery: control of the resources of a whole nation).

3. Achieving low rejection rate.
Unless proper incentives are established, asking citizens to spend some significant time and effort in order to reach a decision or a set of decisions may yield few takers. In fact, in applications of sortition in recent years the rate of acceptance has usually been under 10 per cent. Of course, convening a body made of the minority of invitees who accept will result in a biased sample—yet this is essentially the standard practice. (This has usually been combined with a quota system which allocates representation to some pre-selected groups in the population, e.g. gender groups and age groups, according to their proportion in the population.) It has been suggested that the way to handle low acceptance rates is by making service mandatory, as is the custom for jury duty. However, this may not always be an option and, in any case, having a decision-making body where most members have been pressed into service seems like a recipe for disaster.

Get involved

If these issues, and other relevant issues, seem interesting and important to you, and you’d like to take part in writing guidelines for how they should be properly addressed, please contact me at yoram_gat@yahoo.com and become part of a working group on this topic. Please also forward this invitation to any statistician who may be interested.