Deliberation May Improve Decision-Making

By Neil_Dullaghan🔹 @ 2019-11-05T00:34 (+66)

Many in the effective altruism community are concerned about improving institutional decision-making,^[1] and even more are concerned with the lives of future generations and non-human animals who are currently not represented in our decision-making systems.^[2] One approach to improving decisions is to set up institutional structures that are conducive to good decision-making (Thaler & Sunstein, 2008; Whittlestone, 2017a). A range of recent proposals have been made to reform political institutions towards better decision-making e.g. approval voting, age-weighted voting, future generation orientated offices and committees,policies addressing Transformative AI and many more. Reforms to increase deliberation in the decision-making system is another potential option to consider. Deliberation, the process of weighing the merits of competing arguments with at least the aim to arrive at a consensus, can lead to more analytical reasoning, and under the right circumstances can lead to future-oriented thinking. Such improvements to decision-making may be especially useful in complex high-stakes decisions where it’s difficult to even know what the best decision is, let alone actually making that decision (Whittlestone, 2017).

In this piece, we discuss the opportunities that deliberative reforms offer for improving outcomes in institutional decision-making. We begin with a discussion of deliberation and its links to democratic theory, and then sketch out examples of deliberative designs. Following this, we explore the evidence that deliberation is possible and what direct and indirect effects it may have. It appears that deliberation can engender more fact-based reasoning, improve decision-making, and can shift opinions in favour of future generations. So far, most deliberative initiatives have not been invested with a direct role in the decision-making process and so the majority of effects we see are indirect. Providing deliberative bodies with a binding and direct role in decision-making could improve this state of affairs. We end by highlighting some limitations and areas of uncertainty before noting who is already working in this area and avenues for further research.

Democratic deliberation

Deliberation is a process distinct from open-ended discussion or zero-sum argumentation because it involves a collective exchange of reasons among equals _as if _ they aim to reach a shared practical judgment. This is quite unlike negotiation or adversarial debate where one side is trying to “win”. In contrast to compromise, a genuine consensus is not simply a negotiated agreement of the participants, but includes a transformation of preferences. Even if agreement is not reached participants are more likely to leave a deliberation as more sophisticated, tolerant, and participative citizens who are better able to process more information more objectively (Gastil, Deess, & Weiser, 2002; Gastil & Dillard, 1999; Tetlock, 1983, 1985). This is due to the environment in which participants are exposed to information. As all arguments are given a fair hearing, participants are more likely to carefully consider competing arguments and gain experience in better decision-making habits such as understanding expert opinions. It can expand the information basis of decision-making and enhance the level of reflection among the participants.

Many of the deliberative reforms discussed below have been proposed by democratic theorists. Democratic theory took a “deliberative turn” in the 1990s, shifting focus away from territorial representation and maximizing citizens’ expression of political preferences and towards randomized representation, akin to jury selection, and to maximizing citizens’ capacity to form preferences and judgments on public affairs. The outcome of this shift is a desire for discursive or deliberative democracy. Deliberative democracy is “a system of self-government that concerns itself as much with the quality of its internal deliberation as it does with the distribution of formal power” (Gastil & Richards, 2013, p. 255). Here we are less concerned with the overall structure of democracy but rather the act of _democratic deliberation, _which ties together the egalitarian, open character of the forum and rigorous analytic decision-making. Such deliberation can occur both in democratic and non-democratic regimes, being notably popular in China (He & Warren, 2011; Ma & Hsu, 2018).

From the perspective of deliberative democrats, deliberation may offer a way to fill the gap between elites who appear unresponsive to the public’s concerns and populists who use emotive non-deliberative appeals. Ideally, it fills this gap with a reasoned weighing of arguments that represents the interests of the wider public. Of course although there are democratic reasons to be concerned about appropriate deliberation, the broader idea that many EAs may be concerned with is that it will improve decision-making.

Is deliberation possible?

A key component of deliberation is the weighing of competing arguments and deciding on the merits of the argument with good information. Is there evidence that individuals can actually deliberate in an effective way i.e. weigh trade-offs, assess competing arguments, and connect deliberations with decisions. Is such a thing possible in an era of fake news, social media echo chambers, and ideologically sorted communities? It may not be important for many EAs that deliberation meet exactly the standards of democratic theorists, so long as better decisions result from the process. However, it is an advantage of deliberative methods over some other proposals that increasing deliberation is arguably democratically legitimate and explicitly directed at improving reasoning.

Empirical research shows that both politicians and average citizens have the capacity to deliberate when institutions are appropriate. The Discourse Quality Index is a content analytical measure for capturing the quality of deliberative processes. The quality of deliberation is evaluated by (a) the extent to which deliberation fulfills a number of vital characteristics ascribed by deliberative theory and (b) whether deliberative behavior is equally distributed among the participants. Despite reasonable concerns that politicians may be too partisan and short-sighted to deliberate openly, the Discourse Quality Index offers evidence that political discourse can meet the ideals of deliberation as envisaged (Steenbergen, Bächtiger, Spörndli, & Steiner, 2003). This is especially so in settings of coalitions, second chambers of parliament (for example, the US Senate or UK House of Lords), secrecy, low party discipline, low issue polarization, and the strong presence of moderate parties (Fishkin & Mansbridge, 2017, p. 10). However, combinations of these conditions (e.g. a less polarized issue debate in a second chamber of a consensus system behind closed doors) may represent relatively rare conditions and ones that are harder for EAs to generate.

Ordinary citizens are also capable of this level of discourse in deliberative environments. For example, Europolis, a European-wide Deliberative Poll, provides evidence that citizens are able to reason in ways comparable to those of the parliamentarians. The poll brings together a sample of the voters of the entire European Union, representing all 27 countries for a three-day deliberation before the European Parliament elections. Participants discuss the issues, read balanced briefing materials, question competing experts and politicians, and register their opinions and voting intentions in confidential questionnaires. The number of participants who both provided a sophisticated justification and engaged in respectful listening is almost 30 percent of those sampled (Gerber, Bächtiger, Shikano, Reber, & Rohr, 2018). Whether or not this is high is up for debate. This public deliberation has been found beyond Western countries with high literacy and high education levels e.g. Uganda (Fishkin et al., 2017) and China (Fishkin, He, Luskin, & Siu, 2010).

Some cast doubt on the positive effects of deliberation. For example, Hibbing & Theiss-Morse argue

real life deliberation can fan emotions unproductively, can exacerbate rather than diminish power differentials among those deliberating, can make people feel frustrated with the system that made them deliberate, is ill-suited to many issues and can lead to worse decisions than would have occurred if no deliberation had taken place” (2002, p. 191).

Their methods have been criticised however, because they ignore contrary evidence in their own survey data, and they use the conclusions of focus groups, who are in effect deliberating, to show that citizens do not want to deliberate (Dryzek, 2007). Allaying fears that discussions between like-minded people can shift them towards more extreme opinions, there is no tendency for groups to become more polarized as an outcome of group deliberation (Luskin, Fishkin, & Jowell, 2002) due to the explicit inclusion of opposing points of views in deliberative forums.

How do we design deliberative systems?

A “deliberative design” is any number of processes that provides a framework wherein a participant defends a view by providing reasons; others probe the usefulness of this view through criticism; by reflecting together on the evidence for and against various views, free and independent participants come to accept what Habermas calls “the force of the better argument” (Habermas, 1984, 1996). Common elements of the deliberative designs discussed here are the random selection of participants, hearing of experts, access to written materials, and deliberation leading to “verdicts” on (policy) issues.

Researchers have found three conditions that tend to motivate individuals to adopt a deliberative frame of mind: accountability (Tetlock, 1983, 1985), high stakes (Taber, Lodge, & Glathar, 2001), and diversity of perspectives (Mutz, 2002a, 2002b; Mutz & Martin, 2001; Nemeth, 1986). Therefore, deliberative designs should ensure that participants know that they will have to discuss their judgments publicly, they perceive that there will be consequences, and that a range of perspectives are represented. Beyond this there are a plethora of ways in which deliberation can be designed depending on one’s objectives and goals.

Existing political institutions and designs may allow for deliberation, but there are a number of reasons to doubt that they can maximise it. Negotiation rather than deliberation appears to be the norm among political elites, and so changes in opinion are more likely to be the result of changes in circumstances (which may be harder to influence) rather than the effect of weighing the merits of competing arguments (Fishkin & Luskin, 2005). To the extent that one believes politics is inherently an adversarial competition, it is more difficult to imagine deliberative cooperation as a tractable proposal. Elected officials are also likely to have short-term biases because of the pressures of electoral cycles and so may not be the best agents to advocate for better longterm outcomes. As noted above, the structural conditions that appear to facilitate greater deliberation among politicians seem hard for EAs to influence.

In theory, policies influenced by deliberative forums will have greater legitimacy if they are derived from citizens and/or are supported by citizen groups. Therefore, another avenue of deliberative reform is to use random selection to create “mini-publics” (Fung, 2003) drawn from the population. The rationale for mini-publics is that if the random sample of citizens that is gathered to deliberate is representative of the population, and if it deliberates under good conditions, then its considered judgments after deliberation should represent what the larger population would think if somehow those citizens could engage in similarly good conditions for considering the issue. This is a somewhat idealized notion since we can easily imagine that the result of actually gathering a sufficiently large population to deliberate would be chaotic. For our purposes it may only matter that mini-publics are legitimized because of this belief rather than due to the reality of such a claim.

Sortition, the random selection of participants, is the preferred method for deliberative forums. Firstly, the existence of a new elected body could undermine the legitimacy of other existing elected bodies. Secondly, electing individuals to participate runs the risk of representatives not being sufficiently future-focused due to short-term electoral incentives. It is possible that unlike politicians and interest groups, randomly selected citizens are unlikely to have the interest or the capacity to entrench themselves in their public role of deliberators, and in any case short term limits as deliberators can limit the development of either. Thirdly, sortition also avoids the issue that self-selected participants will come from a homogeneous group (e.g. mostly white, college-educated, and middle-class in North America and Europe) and this homogeneity may impede a deliberative frame of mind. Some scholars are skeptical that we can achieve ideal randomisation (Ryfe, 2005), and indeed, it is rare for such sortition to meet standards of statistical representativeness because of how deliberative designs are implemented in practice. Low response rates, insufficient or absent incentives for participation, the imposition of demographic quotas for political purposes, and weak or improper use of stratified random sampling can lead to mini-publics that do not truly reflect a wide range of opinions in society nor their proportion. There is always some element of self-selection despite designers’ best intentions.

Examples of deliberation among randomly chosen citizens can be found in classical Athens, Medieval Italian republics and among modern iterations begun in the 1970s with the creation of citizens’ juries and planning cells (Carson & Martin, 1999; Manin, 1997). Depending on the definition one uses there can be hundreds of deliberative designs.^[3] The number of participants can range from 25 to 400 and can be conducted in just a few days to over the course of multiple weeks. We will highlight here just some illustrative examples of the most interesting and most researched types of deliberative forums.

• Deliberative polling brings together a random sample of citizens, explicitly asserting statistical representativeness (Mansbridge, 2010), for a weekend of small group discussions with trained experts and gathers data, and survey them as in an opinion poll both upon recruitment and then again at the end of the deliberations. Finally, the results of the polls are released to media outlets. Regularly run deliberative polls now exist such as Europolis in the EU and Stanford’s Deliberative Polling across the world. This has been proposed as a way to understand public opinion on AGI.
• Design panels evaluate and potentially revise proposed ballot measures before being circulated for signatures. These may be of particular interest to EAs considering ballot initiatives as a tool. It offers a quasi seal of approval that can provide inexpensive credibility to a well-crafted, but underfunded, ballot measure and reduce the legally required signature threshold for one (Gastil & Richards, 2013, p. 268).
• Citizens’ Assemblies forward a recommendation to the legislature whereby its proposal goes directly to the ballot unless the legislature introduces a measure to block it. This is ideal for addressing issues on which the legislature has a conflict of interest such as term limits and campaign reform. For example, the 2004 British Columbia Citizens’ Assembly, and proposals for Citizens’ Assemblies on Brexit to address the generational fallout of such decisions. Scotland has organised a citizens’ assembly on how to respond to the UK’s exit from the EU. One could also aggregate the conclusions of many small citizens' assemblies or mini-publics to form a collective mass opinion similar to what AmericaSpeaks does.
• Citizens’ Initiative Reviews develop a one-page analytic statement (typically about a ballot measure) that appears in the Voters’ Guide. For example, Oregon Citizens’ Initiative Review
• Policy Juries. In this proposal (Leib, 2004, p. 12), a stratified random sample of fifty citizens deliberates for two weeks on a specific piece of legislation. The judgment is final and subject only to judicial review. This is probably best suited to addressing issues on which the unconsidered opinion of the mass majority opinion is suspect, such as the civil rights of minorities.
• Consensus Conferences. Denmark designed these in the 1980s to solve problems on issues characterized by technical and moral complexity. They are now being convened in numerous countries.This design may be especially useful for issues surrounding biotechnology/security.
• Proposals for a non-partisan legislative house, chosen by sortition, given the power to scrutinise and if necessary to veto legislative proposals e.g. A Council of Guardians of Future Generations (Read, 2012) or a “tribunate assembly” (McCormick, 2006, p. 160)
• Planning Cells. A number of cells of 25 randomly-selected citizens meet over a few days and are provided with information on a policy issue. They deliberate and prioritise potential courses of action. Moderators prepare a final report for the commissioning body. Planning cells have been used over 170 times in more than 40 locations, being common in Germany for decades.

Lafont (2017) offers a framework of when to use different mini-publics depending on how different the opinions of the mini-public and mass-public are likely to be. If the opinion of the mini-public will likely differ from the majority opinion of the population on the political issue at hand (e.g. the historical marginalisation of one group), then their conclusions can act as a powerful tool for minorities to challenge the status quo. For example, mini-publics could be routinely convened to provide the Supreme Court (or relevant judicial body) with additional, supposedly unbiased, information on what the considered majority opinion of the country may be at a given time.

If the mini-public’s recommendations are likely to coincide with the majority opinion but differ from existing policy, it can signal to the wider public the need to scrutinize the political system. Regular polling could rank important political issues that need to be tackled, and then mini-publics could be convened to make recommendations concerning the top-ranked issues. This would be particularly helpful concerning political issues that elected officials may see as intractable or not worth confronting. We might think that Citizens’ Assemblies here could tackle electoral reforms, while Consensus Conferences could tackle highly technical issues where exiting government policy is lagging behind such as biosecurity or AI.

If the public has no opinion at all about the political issues in question (e.g new gene editing technologies such as CRISPR or international trade agreements) then the function of mini-publics would not be to directly shape the policies in question, but instead to enhance the visibility of what is at stake so as to enable public debate among citizens. Design Panels and Citizens’ Initiative Reviews could perform this role for ballot initiatives tackling issues EAs are interested in but are not yet subject to much public discussion or partisanship.

How likely are these reforms to be supported by the public and the political elite?

Interviews with Belgian mini-public participants reveal that they conceive mini-publics as a complement to electoral forms of participation, like voting or party membership (Jacquet, 2019) and British opinion poll data suggests “not only are people prepared to join ‘juries’, but the public at large is willing to trust their decision-making – even over that of elected representatives” (Lowndes, Pratchett, & Stoker, 2001, p. 448). Countries with higher levels of deliberation in government are also more likely to have support for the regime (Seitz & Votta, 2018). A novel two-study design on awareness of mini-publics aimed to ensure replicability and is notable for using real information for its vignettes (Boulianne, 2018). Both studies found that being informed about mini-publics positively affected respondents’ perceptions that they could influence government and the government cares about their opinions. The first study also found that being informed that policy options were recommended by a randomly selected group of citizens who met to discuss different policy options (a mini-public) had an impact on some policy preferences but this was not replicated in the second study. The explanation given is that the second study used information about mini-publics tackling different policy issues. The author’s conclusion was that citizens can discriminate between a mini-public’s recommendations depending on the policy domain and choose whether to accept the recommendation or not. How successful such policy recommendations have been in practice is dealt with in the Policy Influence section below.

There are a number of routes through which these reforms have come to be implemented by politicians. The Oregon Citizens’ Initiative Review was the result of a pilot test run by an NGO, Healthy Democracy, that caught the attention of the Secretary of State who then recommended it to the legislature. The citizens assembly in British Columbia came as the result of an electoral promise, and Denmark's consensus conferences were established by the Danish Board of Technology, an independent institution set up by the parliament.

Impact of Deliberation

We have established above what deliberation is, how one could design deliberative bodies, and that randomly chosen citizens are likely capable of deliberation. However, does this deliberation actually lead to better outcomes?

Better Decision-Making

Do institutions with more deliberation generate better decision making? There is a gap in the empirical literature on the relationship between measures of deliberation and outcomes at the state-level. Most studies focus on where and whether deliberation is occuring, rather than what effect it has.

A starting point could be to consider that consensual democracies, such as the Nordic countries, the Netherlands, Germany, and Switzerland are arguably the world’s most successful states on a variety of indicators, suggesting a strong correlation between deliberation and public policy success. One of the few studies addressing this question looked at four national legislatures (Switzerland, United States, Germany, and the UK) and found more unanimous decisions are typically associated with higher Discourse Quality Index scores, however they did not find evidence that high discourse quality increases egalitarian decisions, in the sense that the most disadvantaged in society are particularly helped (Bächtiger, Spörndli, Steenbergen, & Steiner, 2005). It may be that actual instances of deliberation are so rare and so overpowered by the dynamics of power politics in existing institutions that there indeed is no effect in practice.

<p id="gdcalert1" ><span style="color: red; font-weight: bold">>>>>> gd2md-html alert: inline image link here (to images/Deliberation-May0.png). Store image on your image server and adjust path/filename if necessary. </span><br>(<a href="#">Back to top</a>)(<a href="#gdcalert2">Next alert</a>)<br><span style="color: red; font-weight: bold">>>>>> </span></p>

(A world map indicating the Deliberative Component Index score (normalized to range from 0 to 1) across the countries. Data weighted to same sample size (=1000). (Seitz & Votta, 2018, p. 17))

However, the Discourse Quality Index best suited to assessing and comparing the quality of deliberation in actual speech acts/debates but not across whole political systems. The only available measurement of deliberation on the structural level that serves the purpose of quantitative cross-national comparison is the Deliberative Component Index of the “Varieties of Democracy” Project (Coppedege et al., 2017) but few studies have employed it to look at policy impacts as far as we know. This measure is based on expert-based assessments of the extent to which political elites give public justifications for their positions on matters of public policy, justify their positions in terms of the public good, acknowledge and respect counter-arguments, and how wide the range of consultation is at elite level. To the extent that one regards trade liberalisation as a positive outcome, countries that score higher on the Deliberative Component Index are negatively associated with trade tariffs and positively associated with trade to GDP ratio (Singh, 2018). More studies in this area would be a welcome addition to the literature. In the meantime, we can look to micro and meso level evidence of specific deliberative initiatives.

Deliberative Polls have been run in countries around the world since the 1990s. Studies based on these deliberative polls regularly report that people who engaged in deliberation were more likely to have clear and ordered policy preferences(Farrar et al., 2010) and opinion changes between pre- and post-treatment questionnaires, at both aggregate and individual levels (for a detailed overview see Setälä & Herne, 2014).

A standardised effect size comparison or rigourous meta review does not appear to have been done yet so here are just some illustrative examples to show the effects on opinion change and increases in subject knowledge. A Danish national Deliberative Poll on the single European currency found 28% of the participants altered their views from ‘agree’ to ‘disagree’ or vice versa (Andersen & Hansen, 2007). The average percentage of respondents giving correct answers to various statements and questions about the single currency improved 22% among those in the deliberation, compared to 6% in the control. The result of the Deliberative Poll was a majority for Denmark joining the single currency. In contrast, in the subsequent referendum on the single currency, a majority of Danes decided against Denmark's participation. In a deliberative forum in Arizona (Barabas, 2004) citizens who attended nearly halved their proportion of incorrect responses about factual information on the Social Security program and were less likely to offer incorrect or don’t know responses. Forum participants also increased their support for raising the amount of earnings subject to Social Security payroll taxes from 60% to 72%, compared with a small decrease for the control group. Significant opinion changes on policy issues as a result of deliberation have been found to endure even six to nine months after deliberation (Boulianne, Loptson, & Kahane, 2018; French & Laver, 2009).

Some studies of deliberative polls have found little or no support for the proposition that participation in deliberation has significant effects on knowledge, sophistication or opinion change. However, there are often issues with the data or deliberation. For example, the data in a study in the UK (Denver, Hands, & Jones, 1995) are not sufficiently reliable to support very detailed analyses of the hypotheses proposed largely because of participants dropping out between pre- and post-surveys.

<p id="gdcalert2" ><span style="color: red; font-weight: bold">>>>>> gd2md-html alert: inline image link here (to images/Deliberation-May1.jpg). Store image on your image server and adjust path/filename if necessary. </span><br>(<a href="#">Back to top</a>)(<a href="#gdcalert3">Next alert</a>)<br><span style="color: red; font-weight: bold">>>>>> </span></p>

(A Deliberative Poll of over 200 California public high school seniors. Credit: Center for Deliberative Democracy )

It is possible that these effects may be mainly due to information material and interaction with experts, and unfortunately many of the studies fail to disentangle the effects of the given information from that of deliberation. While a deliberative poll of a representative sample of EU citizens found that, compared with a control group, deliberators changed their views significantly on immigration (becoming more liberal), climate change (becoming greener) and the EU itself (becoming more pro-European), none of the explanations they tested were satisfactory: sampling bias, increased political knowledge, discussion quality, small group social conformity pressure and the influence of other Deliberative Poll actors (Sanders, 2012). However, results from a randomized field experiment suggest deliberations themselves drove most of the change, rather than other aspects of the Deliberative Polling treatment such as the invitation, briefing materials, or casual conversations with other attendees (Farrar et al., 2010). For consequentialists it may not matter whether it is their deliberation or their contact with experts that is producing the results, so long as the results are present.

The Longterm Future

From a democratic perspective, it may be illegitimate to make decisions that do not adequately represent the interests of those affected, including future generations (Goodin, 2007; Tännsjö, 2007). Even leaving aside democratic ideals, others may regard it as unjust to make decisions that ignore or violate the interests or rights of future generations (Barry, 1978). Deliberation may help to motivate longterm thinking and so may appeal to temporal cosmopolitans in EA. A number of scholars have suggested that there may be a relationship between deliberation and longterm thinking about future generations (mainly concerning issues around the environment and climate change, affecting 50 years in the future) (Bovenkerk, 2015; Dietz, C. Stern, & Dan, 2009; Ekeli, 2009; Gundersen, 1995; Niemeyer & Jennstål, 2016).

Michael Mackenzie has written a large piece on deliberation and longtermism (MacKenzie, 2018). He argues that the analytical thinking and public reasoning required by deliberation can overcome our cognitive biases, which are often against consideration of the future. In this environment it is easier, and even advantageous for arguments to be made on behalf of future generations because appeals to children and future generations are likely to get a fair hearing (even if they are made by those not trying to genuinely represent the interests of the future). If this were to be done at a large scale it may foster coordination between non-overlapping generations and protect against time-inconsistency problems. This view is based on the assumption that our disregard for the future is mainly the product of our cognitive biases rather than just selfishness and a reluctance to expand our moral circle. More research in understanding the causal mechanism here would be interesting but is not necessary so long as better outcomes for future generations are produced as a result of deliberation.

Mackenzie suggests three conditions are necessary for deliberation to generate longterm thinking. First, collective decisions on those issues must be based on genuine deliberation (rather than adversarial debate or negotiation one assumes). Second, the interests of the future will not be relevant in deliberation unless we recognize that our decisions will affect the future (achieving what Niemeyer & Jennstål (2016) elsewhere call a ‘deliberative stance’ when a cognitive dimension is present that reflects upon the consequences of decisions). Thirdly, there must be some disagreement about how different groups will be affected by the decisions taken on longterm issues. Others would add that the current generation’s decision to benefit the future generations will be dependant on their level of altruism (Kamijo, Komiya, Mifune, & Saijo, 2017). Niemeyer & Jennstål (2016) in an earlier proposal offered a similar framework but also included that future generations must be ‘emotionally present’ in that process, and not abstract or statistical entities. We cannot assume that deliberation will automatically include future generations if it is not embedded in the design and agenda, but it may still benefit them to the extent that it generates less biased judgement, and more stable decisions and institutions post-deliberation. For our purposes we may not be concerned that those deliberating actually engage in longterm thinking so long as their decisions have positive longterm outcomes.

Some fascinating work that provides data in support of MacKenzie’s hypothesis that people can be guided into longterm thinking by a prompt to consider future generations has been done in Japan and is worth discussing in some detail. One study shows the pro-longtermism effect of introducing a negotiator on behalf of future generations, and the other of having one group imagine themselves as future generations.

In the first (Kamijo et al., 2017), a laboratory-controlled intergenerational sustainability dilemma game (ISDG) shows how the presence of negotiators for a future generation increases the benefits afforded to future generations. Participants were divided into two groups, which were then further divided into teams of 3 who represented one of 5 concatenate generations. Each generation was presented with option A and option B on how to divide money. An essential feature of the ISDG is that the choice of the current generation affects the size of the next generation’s money. Option A brings a larger benefit to the current generation, but it is detrimental to the benefit of the next generation. Option B brings less benefit to the current generation, but preserves the size of the pie as it is. All generations obtain the same amount when they continue to choose Option B, but if they continue to choose Option A their resources shrink gradually.

An important element here is the treatment condition. In one of the two groups, one of the members in each generation making the choice acts as a negotiator on behalf of future generations. When faced with members of an imaginary future generation, 60% of participants opted to leave resources for future generations (choosing Option B), even if that meant reducing the remuneration which the group itself would take home. In contrast, when the imaginary future generation was not salient, only 28% of participants chose the sustainable option. Additionally, the imaginary future generations, as well as other members (i.e. not-imaginary-future-generation members) in the treatment condition, produced more positive statements on a sustainable option than participants in the control condition. These games do not necessarily entail deliberation since only 10 minutes were allotted for discussion but the idea was that people in the same generation (should) discuss and take a decision as a group. The same experiment was conducted in Dhaka (an urban community) and a rural community in Bangladesh as well as in Kathmandu (an urban community) and a forested community in Nepal; in nonurban communities the results replicated the positive benefits to future generations of a future negotiator, but no effect was detected in urban communities (Shahrier, Kotani, & Saijo, 2017).

In an example much closer to genuine deliberation (Hara, Yoshioka, Kuroda, Kurimoto, & Saijo, 2019), in Yahaba Town, Japan, a series of workshops divided participants into future and present generation groups to first deliberate separately and then together in order to form a consensus over prioritizing policy measures for 2060. Each workshop lasted 2.5 hours and participants had access to Town Hall staff members, university faculty members, and research materials. As a condition for becoming a future person of 2060, the participants were asked to assume that they had time-traveled to the year 2060 without aging (i.e., they were of the same age in 2060 as at the present). The members of the imaginary future-generation groups wore special Yahaba Town happi coats to help them identify as part of the imaginary future generations.

<p id="gdcalert3" ><span style="color: red; font-weight: bold">>>>>> gd2md-html alert: inline image link here (to images/Deliberation-May2.jpg). Store image on your image server and adjust path/filename if necessary. </span><br>(<a href="#">Back to top</a>)(<a href="#gdcalert4">Next alert</a>)<br><span style="color: red; font-weight: bold">>>>>> </span></p>

(Yahaba Town residents in modern clothes and future generation clothing. Credit: Ritsuji Yoshioka )

The present-generation groups drew up a vision as an extension of the status quo, taking the constraints and challenges that exist today as given, while the future-generation groups called for efforts to address tough challenges in order to enhance the strengths of the town. When both groups deliberated together the consensus arrived at included more than half of the measures originally proposed by the imaginary future-generation groups (in many cases being ideas that were absent from those produced by the present-generations group). This study is notable for using a real world setting. All of the visions and measures identified throughout the workshop were crafted for inclusion in a policy document prepared by the town to address Japanese government policy.

The two examples above seem to counter doubts that deliberation on behalf of future generations would be distorted and detrimental (Jensen, 2015, p. 541). Longterm thinking is not always successfully achieved however. Niemeyer & Jennstål (2016) highlight a number of examples where future generation orientated deliberation was attempted but failed to produce notable effects.^[4] The authors put this down to poor deliberative design including limited time available, the absence of activities facilitating group development, the use of voting on decisions, and a lack of diversity in perspectives of the participants.

Non-Human Animals

Similarly to the case of future generations, it is possible that deliberation allows for arguments on behalf of non-human animals to get a fairer hearing. The deliberative exercises we reviewed had relatively little impact on policy makers, although there was some evidence of an attitude shift amongst the participants, and these tended to be in the direction of support for greater protection for animals. The provision of information may be playing a bigger role in these effects than the deliberation itself, but for our purposes this may not be a crucial consideration.

One case study examines a deliberative forum, the Boyd Group, convened by civil society to find agreement on reducing, if not eliminating, the use of animals in experiments (Garner, 2017). This does not meet all the criteria outlined above since participants represented groups, rather than appearing as individual citizens. As such the group consisted of experts and partisans on the issue. While deliberation had the effect of softening some of the views and attitudes, there is little evidence that deliberation significantly shifted participants’ views on substantive issues. The near consensus emanating from the Boyd Group deliberation was to maintain the status quo of the regulatory process. The few changes that the Boyd Group did recommend (for example, the banning of the testing of cosmetic ingredients, and the ban on the use of the Great Apes) did find their way into the British Government’s programme. However, it is difficult to know whether the Home Office was already moving in the direction recommended by Boyd Group reports.

In the Welfare Quality project,^[5] the aim of the citizens’ juries set up as part of the exercise was to assess citizens’ responses to the farm animal welfare protocols drawn up by animal welfare scientists. It was reported that some jurors were ‘quite shocked and surprised’ by the reality of intensive animal agriculture, and all of the members of the Italian jury, and the vast majority of the UK jurors, admitted they were not aware of the sheer extent of intensification. In particular, they were shocked about the stocking densities in broiler sheds, and the short life-spans of broiler chickens (Miele, Veissier, Evans, & Botreau, 2011, p. 113). The jurors in general were fully supportive of strong animal welfare measures and they were not convinced that the protocols developed by animal welfare scientists went far enough. Opposition to factory farming in the Welfare Quality project occurred because the participants, having possession of the facts about the animal suffering involved, came to the conclusion that this suffering outweighed the human benefits to be gained (for example, the cheap and plentiful supply of meat) from the practice. There was never any question of the deliberators being converted to the view that animals should not be used as sources of food. We do not know if there was a shift in opinion as a result of deliberation because their views at the outset of deliberation were not measured.

A deliberative exercise on animal research was conducted in 2013. It came about as a result of Ipsos MORI (a European polling organisation) being commissioned by the lobbying group Understanding Animal Research (UAR). After hearing factual information and the case for and against animal research, the participants across all of the workshops became more favourably inclined towards animal research (Ipsos MORI, 2013, p. 24). However, when undercover footage of misdemeanors in laboratories was provided ‘many reverted to an oppositional stance in relation to animal research’ (2013, p. 25). As a result, participants were much more willing to consider more rigorous scrutiny including insisting that license applications be subject to external scrutiny, and even that CCTV be placed in labs to be screened in public, an idea that gained ‘much support’ (2013, p. 6, 42-4). However, there was no conversion to the position that animals should not be used as experimental subjects.

There was a ‘consistent lack of support’ for xenotransplantation^[6] in the so-called Deliberative Mapping Project, and it was the worst performing option (out of nine) across all of the Citizens’ Panels (Eames et al., 2004a). Participatory Technology Assessment exercises in the Netherlands and Canada recommended a moratorium on xenotransplantation, whereas a minority in the Swiss Participatory Technology Assessment did so too (the majority opting for regulation) (Griessler, 2011, p.38). In the Deliberative Mapping Project too, many held a negative view of xenotransplantation from the start but it also ‘suffered significant negative shifts in performance’ particularly after the joint workshop when members of the Citizens’ Panels were exposed to the views of the expert (Eames et al., 2004b, p. 39). However, animal welfare was never the only, or the most important, issue in any jury.

Policy Influence

It may be all well and good if deliberation can improve the decision-making of its participants, but unless we aim to convene the entire population in a random assembly (or target small groups of key decision makers) then it is important to see if these mini-publics have flowthrough effects on policy stemming from the conclusions deliberations reach.

Direct Influence

Democratic theorist Dryzek has noted “direct influence on and in policy making is a hard test for mini-publics to pass. While examples exist of influence and impact, they are outnumbered by cases where a mini-public is established but turns out to have little or no effect on public decision-making” (Dryzek, 2012, p. 170). While seemingly rare, there are cases where deliberative forums have been given a role in the decision-making process.

Oregon Citizens’ Initiative Review evaluated two ballot measures appearing in the state's general election and their conclusions were included in voter guides sent out to the electorate. Through a pair of statewide surveys (one rolling cross-sectional and one online panel survey), researchers found that Oregon voters who read the Citizens’ Initiative Reviews’ statements became more knowledgeable about both measures and that they were helpful in deciding how to vote on the issues that the mini-public panels studied. However, a majority of Oregon voters remained unaware of the process and did not read the statements. Following the pilot study the Oregon legislature created the Citizens’ Initiative Review Commission in 2011. When repeated in 2012 however, 52% of Oregonians who completed their ballots were aware of the Citizens’ Initiative Review, and preliminary analyses again showed clear signs of influence on voter decision-making (Gastil & Richards, 2013). A review of Citizens’ Initiative Review pilot tests around the USA in 2018 found that voters found the output useful and “at least two in five said it would make them more likely” to vote on the issue it addressed and may have increased voter knowledge- in some cases “increase respondents’ accuracy rate by 10%—roughly the equivalent of rising one grade level if taking a true-or-false knowledge exam” (Gastil, 2019, p. 21)

Following deliberation, the British Columbia Citizens’ Assembly made a recommendation on electoral rules that was put to the voters in a referendum. The ballot ultimate lost because policymakers required a minimum of 60% support in provincial referenda before they would commit to the recommendations. Similarly to the limited effects of Oregon's Citizens’ Initiative Review, the failures were attributed to a lack of public awareness and knowledge (Fournier, Kolk, Carty, Blais, & Rose, 2011). However, having its recommendations considered and rejected is importantly different from having them ignored altogether, which seems more likely for purely advisory recommendations of mini-publics.

<p id="gdcalert4" ><span style="color: red; font-weight: bold">>>>>> gd2md-html alert: inline image link here (to images/Deliberation-May3.jpg). Store image on your image server and adjust path/filename if necessary. </span><br>(<a href="#">Back to top</a>)(<a href="#gdcalert5">Next alert</a>)<br><span style="color: red; font-weight: bold">>>>>> </span></p>

(British Columbia Citizens' Assembly on Electoral Reform. Credit: Stuart Davis, Vancouver Sun)

The Irish Constitutional Convention gave its recommendations on a list of proposals for constitutional reforms and it could propose any constitutional amendment it wished to be considered. It recommended legalizing gay marriage, which was introduced by public referendum in 2015. The government committed itself to responding to the recommendations of the Constitutional Convention in a timely fashion. This was followed by a Citizens' Assembly which deliberated on issues of abortion, fixed-term parliaments, referendums, population aging and climate change. The Citizens' Assembly recommended ending a constitutional ban on abortion and allowing terminations without restriction until up to 12 weeks of pregnancy. The latter is now law. However, other recommendations of both the Convention and Citizens' assembly were ignored or remain in the legislative grinder.

<p id="gdcalert5" ><span style="color: red; font-weight: bold">>>>>> gd2md-html alert: inline image link here (to images/Deliberation-May4.jpg). Store image on your image server and adjust path/filename if necessary. </span><br>(<a href="#">Back to top</a>)(<a href="#gdcalert6">Next alert</a>)<br><span style="color: red; font-weight: bold">>>>>> </span></p>

(Members of the Irish Citizens’ Assembly vote on the wording of the ballots that were to be subsequently voted on. Credit: Maxwell’s )

Participatory Budgeting in the Brazilian city of Porto Alegre, in which at the lowest of 3 organisational tiers included a set of popular Regional Assemblies open to all, showed “great success in terms of macro-political impact” (Baiocchi, 2001), with budget priorities and representatives to the Forums and Councils being determined by direct vote in these popular Regional Assemblies. However, participants were either self-selected (in the case of the Regional Assemblies) or elected (for the other two tiers) and so do not fit all the criteria.

As discussed above, a movement in Japan called Future Design, led by economist Tatsuyoshi Saijo of the Research Institute for Humanity and Nature in Kyoto, has been conducting citizen assemblies in municipalities across the country. Multiple studies in towns of Yahaba, Suita, Matsumototo have shown that the future residents devise far more radical and progressive city plans compared to current ones.

A Deliberative Poll conducted by What’s Next California in 2011 led to the drafting of Proposition 31, which appeared on the 2012 statewide ballot in California, although it received only forty percent of the vote on Election Day. Deliberative polls have been vested with actual decision-making powers on budgetary matters in some Chinese cities (Fishkin et al., 2010).

Interestingly, it has been suggested that the more deliberative mini-publics are, the less likely they are to influence policy (Curato & Böker, 2016; Graham Smith, Richards, & Gastil, 2015). Mini-publics with low representativeness and low deliberative quality are most likely to produce important policy effects. Furthermore, the more voters knew about the British Columbia Citizen Assembly and Irish Citizen Convention, such as their recruitment mechanisms or their freedom from partisan instructions, the more likely they were to vote for the mini-public’s policy recommendation in the later citizen referendum (M. MacKenzie & Warren, 2012).

While mini-publics have been used in a variety of ways to help inform policy decisions, they have less often been empowered to directly affect policy (Goodin & Dryzek, 2006). Even though the British Columbia and Ontario Citizens’ Assemblies were each empowered to initiate a referendum on electoral reform in their respective provinces, they were not empowered to negotiate policy options with other influential actors (Fournier et al., 2011).

Indirect Influence

The direct political and policy impact of deliberative forums may tend to be low, although this impression is possibly due to there being only a few examples of mini-publics being formally empowered as part of the decision-making process (Goodin & Dryzek, 2006, p. 7). Real impact may have so far come mostly indirectly by working through the broader public sphere, ordinary legislative institutions, and the mass media. Even without being part of the formal decision-making process, they sometimes have an influence on policy. This is often because policy makers take the opinion of people informed in the course of these events as authoritative, in preference to “raw” public opinion and take the option to outsource/delegate the decision for a contentious issue.

Sometimes there will be soft guarantees that recommendations will be acted upon e.g. AmericaSpeaks, Citizens’ Juries, and Consensus Conferences. Indeed, in some cases it does appear they influence legislation. For example, Richard E. Sclove, founding director of the Loka Institute found that

[consensus] conferences that were held in the late 1980s influenced the Danish Parliament to pass legislation limiting the use of genetic screening in hiring and insurance decisions, to exclude genetically modified animals from the government’s initial biotechnology research and development program, and to prohibit food irradiation for everything except dry spices.(Sclove, n.d.)

Of course, it cannot be proven that the Consensus Conference was the decisive influence; skeptics might say this is what government policy could have been anyway.

The Center for Global Development has highlighted a number of examples. In Zeguo, China, a city of about 250,000 inhabitants, deliberative polls helped to channel public spending from high-visibility public works to more basic environmental projects, including a sewage treatment plant. Zeguo has continued Deliberative Polling for budget and infrastructure on a nearly annual basis since 2005 and Deliberative Polling has spread to other areas of China. In Bulgaria, a national Deliberative Poll conducted with the participation of the prime minister led to the end of segregated schools for the historically shunned Roma minority and the closing of Roma-only schools. In Italy’s Regione Lazio, the state that includes Rome, a Deliberative Poll on budgetary issues for health policy helped solve a long-standing problem of too many hospital beds and not enough walk-in clinics. Politicians said the poll results provided cover to do the right thing (Fishkin, 2011). In Germany, a Green-Left government introduced and institutionalized forums for citizen participation and deliberation, subsequently taking up the policy recommendations of those forums (Bächtiger & Beste, 2017, p. 113).

Taking a step away from the political process, eight electric utilities in different parts of Texas commissioned Deliberative Polls between 1996 and 1998, asking customers how they preferred that future electricity requirements to be met. Afterwards utility companies began to integrate such consumer values about energy choices into their decisions (Lehr, Guild, Thomas, & Swezey, 2003). Observers of the poll included elected officials, public utility commissioners, and power company executives, many of whom said in subsequent interviews that watching the deliberation was an important factor in changing their views. The Texas utility commissioner said that subsequent changes in state policy and utility company investment policies were a direct result of the deliberative polls.

Novo Nordisk, a large Danish biotechnology company, reevaluated its research and development strategies after a 1992 panel deplored the design of animals suited to the rigors of existing agricultural systems but endorsed the use of genetic engineering to help treat incurable diseases. The firm moved to concentrate on work more likely to win popular approval, such as animal-based production of drugs for severe human illnesses (Sclove, n.d.).

An Australian consensus conference on GM foods in 1999 was sponsored by the Conumser’s Association, endorsed by government, recognised by anti-GM activists, and allegedly influenced the positions taken by the biotech industry (Hendriks, 2004). However, the policy impact of other consensus conferences on GM food in Denmark, France, and the US, has been harder to show, in part because their recommendations were inline with existing government policy (Dryzek & Tucker, 2008).

In response to a testimony from the Loka Institute a 2003 enactment of the U.S. Congress specified “mechanisms such as citizens’ panels, consensus conferences, and educational events, as appropriate” as ways of “ensuring that ethical, legal, environmental and other appropriate social concerns ... are considered during the development of nanotechnology”.^[7] However, we have not seen much evidence of this being implemented yet.

Reasons to doubt deliberative mini-publics

One of the major claimed advantages of deliberative bodies also reveals a major democratic critique. Deliberation helps participants to make considered judgments that are, because of representative sampling, judged to be what the wider public would share if they were to participate. However, the larger population are left with “unconsidered” opinions. The obvious implication here is that the raw voice of the actual people “is not a voice that by itself deserves any special hearing”(Fishkin, 2013, p. 504) or that the deliberative opinion is worth more. One need not be committed to this view that the raw-opinions do not deserve any special hearing, they might merely be assigned some weight alongside the deliberative views. Selected deliberative citizens groups may also be able to act as representatives of the public (as per elected representatives) even if they don't enact what the public would themselves select. There are also many instances one can imagine where the raw public opinion would not be an input anyway and so mini-publics are not superceding them. It also seems defendable to endorse the outcomes from deliberative mini-publics as legitimate even if we do not believe a deliberation of the entire population would reach the same conclusion. For example, this seems to be the case for juries.

There are concerns that while random sampling prevents overt self-selection, there is still self-selection in who accepts being invited. There is some evidence that people with more education and interest in politics have been found to be more likely to volunteer to participate in mini-publics (Karjalainen & Rapeli, 2015).^[8] Furthermore, even within mini-publics certain people tend to dominate proceedings and amplify inequalities in the wider society from which they are drawn. Studies of Deliberative Polls have found far less distortion than critics expect though (Siu, 2017). More research with controlled experiments could clarify this issue further. Again, one may only be concerned with this issue to the extent that greater representativeness leads to more public and/or elite support for deliberative bodies' conclusions.

It is unclear whether changes in people’s views produced by Deliberative Polls and other consultative mechanisms tried thus far are really improvements on their pre-deliberative views or simply changes. This clearly then depends on what outcomes one judges to be good or bad. With regard to benefiting future generations the examples given above have provided some ideas on how to promote positive longterm changes.

Policy-makers may organize mini-publics in order to strengthen their own position in the eyes of the public, or to advance and legitimize policies they pursue, as sometimes in the case of direct democracy initiatives like referenda. The use of mini-publics could be required when parliaments are legislating on certain types of issues. These could include, for example, constitutional issues, electoral reform and party financing, and perhaps certain types of technically complex matters, which was the case in the Danish practice of consensus conferences.

Some may be concerned that organising such mini-publics imposed too large a financial burden. A 2017 report for the POWER inquiry in Britain suggests costs can range from £16,000 to £200,000 (~$19,500 to $245,000 in 2019 USD) (G. Smith, 2005, p. 51). In particular, televising part of the content through mass media can be very expensive. It can be expensive to get sufficient numbers of participants to create a good representative sample and improving the odds that members of more marginalized groups will attend. Since it's not a duty like jury duty, studies conducted so far require incentives, with costs that have included paying for the trips, the hotel and the food for each participant, and booking an attractive venue; in any case, hiring the research crew and moderators will incur costs. Additional costs have included paying for participants’ compensation so that the people who are randomly selected can put aside their duties to attend the events (e.g., providing child care.). Therefore, if neither the governing body nor other organizations are willing to fund such a poll, there is no way to get it started. However, these still seem likely to be cheaper and easier than other proposed reforms like changing the electoral system.

Groups doing this work

The World Future Council is proposing to establish a deliberative council of Guardians for Future Generations, an independent body that offers advice, recommendations, analysis, and actively advocates for long-term interests .

The Alliance for Future Generations, a group of more than forty organisations and individuals working to bring longtermism and the needs of future generations into the heart of UK democracy and policy processes, want to try to develop an (NGO-led) pilot scheme for local councils of Guardians of the Future Generations (Read, 2012).

The Center for Deliberative Democracy, housed in the Department of Communication at Stanford University, is devoted to research about democracy and public opinion obtained through Deliberative Polling.

The Research Institute for Humanity and Nature & Research Institute for Future Design work on psychology experiments and future generation games in Japanese municipalities, as discussed in the examples above.

Healthy Democracy has continued to conduct biennial Citizens’ Initiative Reviews in Oregon. They have also run statewide Citizens’ Initiative Reviews in Arizona, Colorado, and Massachusetts, in addition to Citizens’ Initiative Reviews at the city and county level. In March 2017, they conducted their first Citizens’ Initiative Review demonstration in California.

Please reach out if you know other groups or people working on this topic and they can be added to this list.

Conclusion

Many of the reforms discussed here are complimentary to other proposals for improving institutional decision-making but there are also some potential advantages of using deliberative bodies in comparison to them. As has been mentioned above, deliberative mini-publics seem to be endorsed by the public as legitimate at least in part because of their composition of randomly chosen citizens. This may offer a more promising route than establishing more (potentially distrusted) elected institutions, technocratic offices, or tinkering with the norms around the weighting of votes. Tasking citizens explicitly with judging whether policies produce the best outcome may be preferable to using proxies for better decision-making, as in the case of age-weighted voting.

This review has also highlighted a number of areas for further research. We have sketched out some examples here of the impact of deliberation but a more thorough quantitative analysis would help in weighing the cost benefit analysis of such reforms. We also need more information on the factors that may influence the impact of mini-publics on public opinion such as the size of the mini-public, length of time spent on deliberation, framing of information received and decisions made, sponsorship of the deliberative event and level of government targeted with the deliberative event. It is too early to know what the long run effects of many of the deliberative bodies will be. It may be informative then to conduct case studies of deliberative forums from the past and consider what the impact of any longterm decisions they made have been.

Credits

This essay is a project of Rethink Priorities. It was written by Neil Dullaghan. Thanks to David Moss, Jason Schukraft and Dominic Roser for comments. If you like our work, please consider subscribing to our newsletter. You can see all our work to date here.

Works Cited

Andersen, V. N., & Hansen, K. M. (2007). How deliberation makes better citizens: The Danish Deliberative Poll on the euro. European Journal of Political Research, 46(4), 531–556. https://doi.org/10.1111/j.1475-6765.2007.00699.x

Bächtiger, A., & Beste, S. (2017). Deliberative Citizens, (Non)Deliberative Politicians: A Rejoinder. Daedalus, 146(3), 106–118. https://doi.org/10.1162/DAED_a_00450

Bächtiger, A., Spörndli, M., Steenbergen, M. R., & Steiner, J. (2005). The Deliberative Dimensions of Legislatures. Acta Politica, 40(2), 225–238. https://doi.org/10.1057/palgrave.ap.5500103

Baiocchi, G. (2001). Participation, Activism, and Politics: The Porto Alegre Experiment and Deliberative Democratic Theory. Politics & Society, 29(1), 43–72. https://doi.org/10.1177/0032329201029001003

Barry, B. (1978). Circumstances of Justice and Future Generations. In R. I. Sikora & B. M. Barry (Eds.), Obligations to Future Generations (pp. 204–48). White Horse Press.

Boulianne, S. (2018). Mini-publics and Public Opinion: Two Survey-Based Experiments. Political Studies, 66(1), 119–136. https://doi.org/10.1177/0032321717723507

Boulianne, S., Loptson, K., & Kahane, D. (2018). Citizen panels and opinion polls: Convergence and divergence in policy preferences. Journal of Public Deliberation, 14.

[Bovenkerk, B. (2015). Public Deliberation and the Inclusion of Future Generations. Jurisprudence, 6(3), 496–515. https://doi.org/10.1080/20403313.2015.1065643](https://www.zotero.org/google-docs/?P4VDST)

Carson, L., & Martin, B. (1999). Random Selection in Politics. Faculty of Arts - Papers (Archive). Retrieved from https://ro.uow.edu.au/artspapers/1944

Coppedege, M., Gerring, J., Lindberg, S. I., Skaaning, S.-E., Teorell, J., Altman, D., & Wilson, S. (2017). V-dem country-year data, version 7.1. Varieties of Democracy (V-Dem) Project. Retrieved from https://www.v-dem.net/en/data/data-version-7-1/

Curato, N., & Böker, M. (2016). Linking mini-publics to the deliberative system: A research agenda. Policy Sciences, 49(2), 173–190.

Dietz, T., C. Stern, P., & Dan, A. (2009). How Deliberation Affects Stated Willingness to Pay for Mitigation of Carbon Dioxide Emissions: An Experiment. Land Economics, 85, 329–347. https://doi.org/10.3368/le.85.2.329

Dryzek, J. S. (2007). Theory, Evidence, and the Tasks of Deliberation. In S. W. Rosenberg (Ed.), Deliberation, Participation and Democracy: Can the People Govern? (pp. 237–250). https://doi.org/10.1057/9780230591080_11

Dryzek, J. S. (2012). Foundations and Frontiers of Deliberative Governance. Oxford University Press.

Dryzek, J. S., & Tucker, A. (2008). Deliberative Innovation to Different Effect: Consensus Conferences in Denmark, France, and the United States. Public Administration Review, 68(5), 864–876. https://doi.org/10.1111/j.1540-6210.2008.00928.x

Eames, M., Burgess, J., Davies, G., Mayer, S., Staley, K., Stirling, A., & Williamson, S. (2004a). Xenotransplantation and its alternatives for addressing the ‘kidney gap’: A case study report from the deliberative mapping project [Report]. Retrieved from http://discovery.ucl.ac.uk/98200/

Eames, M., Burgess, J., Davies, G., Mayer, S., Staley, K., Stirling, A., & Williamson, S. (2004b). Xenotransplantation and its alternatives for addressing the ‘kidney gap’: A case study report from the deliberative mapping project [Report]. Retrieved from http://discovery.ucl.ac.uk/98200/

Ekeli, K. skagen. (2009). Constitutional Experiments: Representing Future Generations Through Submajority Rules. Journal of Political Philosophy, 17(4), 440–461. https://doi.org/10.1111/j.1467-9760.2008.00328.x

Farrar, C., Fishkin, J. S., Green, D. P., List, C., Luskin, R. C., & Paluck, E. L. (2010). Disaggregating Deliberation’s Effects: An Experiment within a Deliberative Poll. British Journal of Political Science, 40(2), 333–347. https://doi.org/10.1017/S0007123409990433

Fishkin, J. S. (2011). When the People Speak: Deliberative Democracy and Public Consultation. Retrieved from https://www.oxfordscholarship.com/view/10.1093/acprof:osobl/9780199604432.001.0001/acprof-9780199604432

Fishkin, J. S. (2013). Deliberation by the People Themselves: Entry Points for the Public Voice. Election Law Journal: Rules, Politics, and Policy, 12(4), 490–507. https://doi.org/10.1089/elj.2013.0200

Fishkin, J. S., He, B., Luskin, R. C., & Siu, A. (2010). Deliberative democracy in an unlikely place: Deliberative Polling in China. British Journal of Political Science, 40(2), 435–448. https://doi.org/10.1017/S0007123409990330

Fishkin, J. S., & Luskin, R. C. (2005). Experimenting with a Democratic Ideal: Deliberative Polling and Public Opinion. Acta Politica, 40(3), 284–298. https://doi.org/10.1057/palgrave.ap.5500121

Fishkin, J. S., & Mansbridge, J. (2017). The Prospects & Limits of Deliberative Democracy: Introduction. Journal of the American Academy of Arts & Sciences, 146(3), 6–13. https://doi.org/10.1162/DAED_x_00442

Fishkin, J. S., Mayega, R. W., Atuyambe, L., Tumuhamye, N., Ssentongo, J., Siu, A., & Bazeyo, W. (2017). Applying Deliberative Democracy in Africa: Uganda’s First Deliberative Polls. Daedalus, 146(3). Retrieved from https://www.amacad.org/publication/applying-deliberative-democracy-africa-ugandas-first-deliberative-polls

Fournier, P., Kolk, H. van der, Carty, R. K., Blais, A., & Rose, J. (2011). When Citizens Decide: Lessons from Citizen Assemblies on Electoral Reform. Retrieved from https://www.oxfordscholarship.com/view/10.1093/acprof:oso/9780199567843.001.0001/acprof-9780199567843

French, D., & Laver, M. (2009). Participation Bias, Durable Opinion Shifts and Sabotage through Withdrawal in Citizens’ Juries. Political Studies, 57(2), 422–450. https://doi.org/10.1111/j.1467-9248.2009.00785.x

Fung, A. (2003). Survey Article: Recipes for Public Spheres: Eight Institutional Design Choices and Their Consequences. Journal of Political Philosophy, 11(3), 338–367. https://doi.org/10.1111/1467-9760.00181

Garner, R. (2017). The Boyd Group and Animal Experimentation. Global Journal of Animal Law, 5(1). Retrieved from https://ojs.abo.fi/ojs/index.php/gjal/article/view/1589

Gastil, J. (2019). Assessment of Three 2018 Citizens’ Initiative Review Pilot Projects. [Report prepared for Healthy Democracy]. University Park, PA: Pennsylvania State University.

Gastil, J., Deess, E. P., & Weiser, P. (2002). Civic Awakening in the Jury Room: A Test of the Connection between Jury Deliberation and Political Participation. The Journal of Politics, 64(2), 585–595.

Gastil, J., & Dillard, J. P. (1999). Increasing Political Sophistication Through Public Deliberation. Political Communication, 16(1), 3–23. https://doi.org/10.1080/105846099198749

Gastil, J., & Richards, R. (2013). Making Direct Democracy Deliberative through Random Assemblies. Politics & Society, 41(2), 253–281. https://doi.org/10.1177/0032329213483109

Gerber, M., Bächtiger, A., Shikano, S., Reber, S., & Rohr, S. (2018). Deliberative Abilities and Influence in a Transnational Deliberative Poll (EuroPolis). British Journal of Political Science, 48(4), 1093–1118. https://doi.org/10.1017/S0007123416000144

Goodin, R. E. (2007). Enfranchising All Affected Interests, and Its Alternatives. Philosophy & Public Affairs, 35(1), 40–68. https://doi.org/10.1111/j.1088-4963.2007.00098.x

Goodin, R. E., & Dryzek, J. S. (2006). Deliberative Impacts: The Macro-Political Uptake of Mini-Publics. Politics & Society, 34(2), 219–244. https://doi.org/10.1177/0032329206288152

Gundersen, A. G. (1995). The Environmental Promise of Democratic Deliberation. Univ of Wisconsin Press.

Griessler, R. (2011) ‘Xenotransplantation Policy and Participatory Technology Assessment in Switzerland’, Institute for Advanced Studies, Sociologial Series 95, Vienna

Habermas, J. (1984). The Theory of Communicative Action. Beacon Press.

Habermas, J. (1996). Between Facts and Norms: Contributions to a Discourse Theory of Law and Democracy. Polity.

Hara, K., Yoshioka, R., Kuroda, M., Kurimoto, S., & Saijo, T. (2019). Reconciling intergenerational conflicts with imaginary future generations: Evidence from a participatory deliberation practice in a municipality in Japan. Sustainability Science. https://doi.org/10.1007/s11625-019-00684-x

He, B., & Warren, M. E. (2011). Authoritarian Deliberation: The Deliberative Turn in Chinese Political Development. Perspectives on Politics, 9(2), 269–289. https://doi.org/10.1017/S1537592711000892

Hendriks, C. (2004). Public deliberation and interest organisations: A study of responses to lay citizen engagement in public policy. https://doi.org/10.25911/5d514726bf726

Hibbing, J. R., & Theiss-Morse, E. (2002). Stealth Democracy: Americans’ Beliefs About How Government Should Work. Cambridge ; New York: Cambridge University Press.

Ipsos MORI (2013) ‘Openess in Animal Research: The Public’s Views on Openness and Transparency in Animal Research.

Jacquet, V. (2019). The Role and the Future of Deliberative Mini-publics: A Citizen Perspective. Political Studies, 67(3), 639–657. https://doi.org/10.1177/0032321718794358

Jensen, K. K. (2015). Future Generations in Democracy: Representation or Consideration? Jurisprudence, 6(3), 535–548. https://doi.org/10.1080/20403313.2015.1065649

Kamijo, Y., Komiya, A., Mifune, N., & Saijo, T. (2017). Negotiating with the future: Incorporating imaginary future generations into negotiations. Sustainability Science, 12(3), 409–420. https://doi.org/10.1007/s11625-016-0419-8

Karjalainen, M., & Rapeli, L. (2015). Who will not deliberate? Attrition in a multi-stage citizen deliberation experiment. Quality & Quantity: International Journal of Methodology, 49(1), 407–422.

Lafont, C. (2017). Can Democracy be Deliberative & Participatory? The Democratic Case for Political Uses of Mini-Publics. Journal of the American Academy of Arts & Sciences, 146(3), 85–105. https://doi.org/10.1162/DAED_a_00449

Lehr, R. L., Guild, W., Thomas, D. L., & Swezey, B. G. (2003). Listening to Customers: How Deliberative Polling Helped Build 1,000 MW of New Renewable Energy Projects in Texas (No. NREL/TP-620-33177). https://doi.org/10.2172/15003900

[Leib, E. J. (2004). Deliberative Democracy in America: A Proposal for a Popular Branch of Government. Retrieved from http://www.psupress.org/books/titles/0-271-02363-5.html](https://www.zotero.org/google-docs/?P4VDST)

Lowndes, V., Pratchett, L., & Stoker, G. (2001). Trends in Public Participation: Part 2 – Citizens’ Perspectives. Public Administration, 79(2), 445–455. https://doi.org/10.1111/1467-9299.00264

Luskin, R. C., Fishkin, J. S., & Jowell, R. (2002). Considered opinions: Deliberative polling in Britain. British Journal of Political Science, 32(3), 455–487. https://doi.org/10.1017/S0007123402000194

Ma, D., & Hsu, S. (2018). The Political Consequences of Deliberative Democracy and Electoral Democracy in China: An Empirical Comparative Analysis from Four Counties. China Review, 18(2), 1–32. Retrieved from JSTOR.

MacKenzie, M. K. (2018). Deliberation and Long-Term Decisions. In The Oxford Handbook of Deliberative Democracy. Retrieved from https://www.oxfordhandbooks.com/view/10.1093/oxfordhb/9780198747369.001.0001/oxfordhb-9780198747369-e-7

MacKenzie, M., & Warren, M. E. (2012). Two Trust-Based Uses of Minipublics in Democratic Systems. In J. Parkinson & J. Mansbridge (Eds.), Deliberative Systems: Deliberative Democracy at the Large Scale. Cambridge: Cambridge University Press.

Manin, B. (1997). The Principles of Representative Government. Retrieved from https://nyuscholars.nyu.edu/en/publications/the-principles-of-representative-government

Mansbridge, J. (2010). Deliberative Polling as the Gold Standard. The Good Society, 19(1), 55–62. https://doi.org/10.5325/goodsociety.19.1.0055

McCormick, J. P. (2006). Contain the Wealthy and Patrol the Magistrates: Restoring Elite Accountability to Popular Government. The American Political Science Review, 100(2), 147–163. Retrieved from JSTOR.

Miele, M., Veissier, I., Evans, A., & Botreau, R. (2011). Animal welfare: Establishing a dialogue between science and society. Animal Welfare (South Mimms, England), 20.

Mutz, D. C. (2002a). Cross-Cutting Social Networks: Testing Democratic Theory in Practice. The American Political Science Review, 96(1), 111–126. Retrieved from JSTOR.

Mutz, D. C. (2002b). The Consequences of Cross-Cutting Networks for Political Participation. American Journal of Political Science, 46(4), 838–855. https://doi.org/10.2307/3088437

Mutz, D. C., & Martin, P. S. (2001). Facilitating Communication across Lines of Political Difference: The Role of Mass Media. The American Political Science Review, 95(1), 97–114. Retrieved from JSTOR.

Nemeth, C. J. (1986). Differential contributions of majority and minority influence. Psychological Review, 93(1), 23–32. https://doi.org/10.1037/0033-295X.93.1.23

Niemeyer, S., & Jennstål, J. (2016). The Deliberative Democratic Inclusion of Future Generations. In I. González-Ricoy & A. Gosseries (Eds.), Institutions For Future Generations (pp. 247–265). https://doi.org/10.1093/acprof:oso/9780198746959.003.0015

Read, R. (2012). Guardians of the Future: A Constitutional Case for Representing and Protecting Future People. Weymouth: Green House.

Ryfe, D. M. (2005). Does Deliberative Democracy Work? Annual Review of Political Science, 8(1), 49–71. https://doi.org/10.1146/annurev.polisci.8.032904.154633

Sanders, D. (2012). The Effects of Deliberative Polling in an EU-wide Experiment: Five Mechanisms in Search of an Explanation. British Journal of Political Science, 42(3), 617–640. https://doi.org/10.1017/S0007123411000494

Sclove, R. (n.d.). Town Meetings on Technology. Retrieved 26 August 2019, from http://www.loka.org/pubs/techrev.htm

Seitz, R., & Votta, F. (2018). Deliberation Across the World—A Cross-National Examination of the Link Between Deliberation and Regime Legitimacy. Conference Paper. Retrieved from https://ecpr.eu/Events/PaperDetails.aspx?PaperID=40475&EventID=115

Setälä, M., & Herne, K. (2014). Normative theory and experimental research in the study of deliberative mini-publics. In K. Grönlund, A. Bächtiger, & M. Setälä (Eds.), Deliberative MiniPublics. Colchester: ECPR Press.

Shahrier, S., Kotani, K., & Saijo, T. (2017). Intergenerational sustainability dilemma and the degree of capitalism in societies: A field experiment. Sustainability Science, 12, 957–967. https://doi.org/10.1007/s11625-017-0447-z

Singh, A. S. P. (2018). Democracy and Trade Liberalisation. Varieties of Democracy Institute: Users’ Working Paper No. 17.

Siu, A. (2017). Deliberation & the Challenge of Inequality. Journal of the American Academy of Arts & Sciences, 146(3). Retrieved from https://www.amacad.org/publication/deliberation-challenge-inequality

Smith, G. (2005). Beyond the ballot: 57 democratic innovations from around the world [Project report]. Retrieved 27 August 2019, from https://westminsterresearch.westminster.ac.uk/item/92y6v/beyond-the-ballot-57-democratic-innovations-from-around-the-world

Smith, Graham, Richards, R. C., & Gastil, J. (2015). The Potential of Participedia as a Crowdsourcing Tool for Comparative Analysis of Democratic Innovations. Policy & Internet, 7(2), 243–262. https://doi.org/10.1002/poi3.93

Steenbergen, M. R., Bächtiger, A., Spörndli, M., & Steiner, J. (2003). Measuring Political Deliberation: A Discourse Quality Index. Comparative European Politics, 1, 21–48. https://doi.org/10.1057/palgrave.cep.6110002

Taber, C. S., Lodge, M., & Glathar, J. (2001). The motivated construction of political judgments. In Cambridge Studies in Political Psychology and Public Opinion. Citizens and politics: Perspectives from political psychology (pp. 198–226). https://doi.org/10.1017/CBO9780511896941.010

Tännsjö, T. (2007). Future People, the All Affected Principle, and the Limits of the Aggregation Model of Democracy 1 In J. Josefsson D. Egonsson (ed.), Hommage à Wlodek. Philosophical Papers Dedicated to Wlodek Rabinowicz.

Tetlock, P. E. (1983). Accountability and the Perseverance of First Impressions. Social Psychology Quarterly, 46(4), 285–292. https://doi.org/10.2307/3033716

Tetlock, P. E. (1985). Accountability: A Social Check on the Fundamental Attribution Error. Social Psychology Quarterly, 48(3), 227–236. https://doi.org/10.2307/3033683

Whittlestone, J. (2017). Two ways of ‘improving decision-making’—Jess Whittlestone. Https://Jesswhittlestone.Com. Retrieved from https://jesswhittlestone.com/blog/2017/9/30/two-ways-of-improving-decision-making

Notes