Science Policy

A Guide to Public Deliberation

06.04.24 | 19 min read | Text by Parmin Sedigh

Science is advancing at an unprecedented speed, and scientists are facing major ethical dilemmas daily. Unfortunately, the general public rarely gets opportunities to share their opinions and thoughts on these ethical challenges, moving us, as a society, towards a future that is not inclusive of most people’s ideas and beliefs. Scientists regularly call for public engagement opportunities to discuss cutting-edge research. In fact, “71% of scientists [associated with the American Association for the Advancement of Science (AAAS)] believe the public has either some or a lot of interest in their specialty area.” Sadly, scientists’ calls often go unnoticed and unanswered, as there continue to be inadequate mechanisms for these engagement opportunities to come to fruition.

To Deliberate or Not to Deliberate

Public deliberation, when performed well, can lead to more transparency, accountability to the public, and the emergence of ideas that would otherwise go unnoticed. Due to the direct involvement of participants from the public, decisions made through such initiatives can also be seen as more legitimate. On a societal level, public deliberation has been shown to encourage pluralism among participants.

Despite the importance of deliberation, it’s important to note that it is not always the best way to engage the public. Planning a public deliberation event — a citizens’ panel, for instance — takes a large amount of time and resources. Plus, incentivizing a random sample of citizens to participate (which is considered the gold standard of deliberation) is difficult. It’s therefore paramount to first assess whether the topic of focus is suitable for public deliberation. 

To assess the appropriateness of a deliberation topic, consider the following criteria (inspired by criteria set forth by Stephanie Solomon and Julia Abelson and the Kettering Foundation):

1. Does the issue involve conflicting public opinions? Issues that involve setting priorities in healthcare, for example, may benefit from public deliberation as there is no singular correct answer; deliberation may offer a more clear and holistic view of what is best for a community, according to the community.
2. Is the issue controversial? If so, deliberation can be a good tool as it brings many opinions into view and can foster pluralism as mentioned previously.
3. Does the issue have no clear-cut solution and is “intractable, ongoing, or systemic”?
4. Do all available solutions have significant drawbacks?
5. Does the community at large have an interest in the problem?
6. Would the discussion of the issue benefit from a combination of expert and real-world experience and knowledge (what Solomon and Abelson call “hybrid” topics)? Certain issues may solely require technical knowledge but many issues would benefit from the views of the public as well.¹
7. Are citizens and the government on the same page about the issue? If not, public deliberation can foster trust, but only if the initiative is done with the intention of taking the public’s conclusions into account.

Setting Goals

If it’s deemed that the topic is suitable for public deliberation, the next step is to set goals for the public deliberation initiative. Julia Abelson, Lead of the Public Engagement in Health Policy Project and Professor at McMaster University, has explained that one of the significant differentiating factors between successful and unsuccessful initiatives is thoughtful planning and organization — including setting clear goals and objectives organizers would like to meet by the end of deliberation. Having an end goal not only helps with planning but also allows for a realistic goal to be shared with deliberation participants. Setting unrealistic expectations as to what the deliberation process is meant to achieve — and subsequently not achieving those goals — will lead participants and citizens, in general, to lose trust in the deliberation process (and organizational body).

Is the goal of deliberation to bring new ideas into view and share those with relevant agencies (governmental or otherwise)? Is the goal instead to enact change in current policies? Is the goal to help shape new policies? The aforementioned Citizens’ Reference Panel on Health Technologies in Canada did not directly impact the government’s decisions, but served to make experts aware of a viewpoint they had not previously explored. This is in contrast to the typical “sit and listen” initiatives that don’t have as much of a capacity to encourage new ideas to emerge. In another instance, a citizens’ jury in Buckinghamshire, England was formed to discuss how to tackle back pain in the county. The Buckinghamshire Health Authority promised to implement the citizens’ recommendations (as was mandated by a charity that was supporting this public deliberation effort) — and they did.

Expanding on the idea of making promises and accountability, it’s important for the organizing body — which may or may not include a federal agency — to consider its role in implementing the conclusions of the deliberation. Promising to implement the conclusion of the deliberations can serve to invigorate discussion and make participants more engaged, knowing that their discussions can have a direct impact on future decisions. For instance, the British Columbia Biobank Deliberation involved a “commitment at the outset of the deliberation from the leaders of a proposed BC BioLibrary (now funded by the Michael Smith Foundation for Health Research) that the Bio-Library’s policy discussions would consider suggestions from this deliberation.” Researchers have suggested this may have contributed to participants’ interest in the deliberation event. Despite some examples of implementation following deliberation (such as the Buckinghamshire and Ontario examples), there continues to be a lack of adequate change based on the public’s recommendations. One other instance comes from NASA’s 2014 efforts to involve the public in the discussion around planetary defense (in the context of asteroids) through a participatory technology assessment (PTA). It seems that the PTA helped to spur the creation of NASA’s Planetary Defense Coordination Office. 

Furthermore, providing updates on implementation to participants, and the public at large, would provide another crucial aspect of accountability: “explanations and justifications.” However, these updates on their own would not fulfill an organization or agency’s duty to accountability as that requires an active dialogue with the public (which is precisely why implementing the conclusions of public deliberation initiatives is important).  

When to Deliberate: Agenda Setting for Citizens

As mentioned above, deliberation can happen at various points during the policymaking pipeline. It has become increasingly popular to include the public early on in the process, such as in an agenda-setting role. This allows the public not only to engage in discussions about a topic but to also set the priorities and frame how the discussions will move forward. As Naomi Scheinerman writes, “with proper agenda setting and precedent creation, the resulting […] questions would be more reflective of what the public is interested in discussing rather than of the companies, industries, and other stakeholder groups.”

A trailblazing model in citizen agenda-setting has been the Ostbelgien Model. The model involves both a permanent Citizens’ Council and ad hoc Citizens’ Panels. Though the members of the Citizens’ Council rotate (and are chosen randomly), one of the permanent roles of the Council is to select topics for the ad hoc Citizens’ Panels, with citizens having a direct hand in what issues their fellow citizens and government should tackle. Since its inception in 2019, the Citizens’ Council has asked Citizens’ Panels to tackle issues such as “how to improve the working conditions of healthcare workers” and “inclusive education.” 

Framing

One of the pillars of the success of public deliberation is a well-scoped question that is framed appropriately. Issues that are framed unfairly, meaning they place emphasis on a specific part of the issue while ignoring others, can lead to inaccurate results and a loss of trust between the public and the organizers. Though this depends on the goals of the deliberation, it’s often best for questions to be specific in their scope to allow for concrete results at the end of the deliberation initiative. For example, an online deliberation session in New York City aimed to assess the public’s views on who should be given priority access to COVID-19 vaccines. One of the questions asked participants to rank the order in which they think a pre-specified list of essential workers should get access to the vaccine. This allows for discussion while retaining a clear focus.

Another example comes from climate change. Climate change can be framed in many ways —  through an economic frame, a public health frame, a justice frame, and others. These various framings impact how the public reacts to the issue; in the case of the economic frame, it has led to “political divisiveness.” Focusing instead on the public health frame, for instance, led to greater agreement on policy decisions. Similarly, according to a 2023 policy paper from the Organisation for Economic Co-operation and Development (OECD), an issue like COVID-19 can be less polarizing if the framing used is about solutions to the pandemic rather than solely vaccines. Importantly, the organizers of the public deliberation initiative do not have sole control over the framing of the issue. Citizens often have a pre-existing “frame of thought.” This makes frames tricky yet essential in making it possible to appropriately and productively deliberate a topic. 

Framing is implicit in that participants in deliberation are not aware of it, making it all the more crucial to be wary of the framing. Thus, it becomes clear how seemingly unimportant factors, such as setting, also affect deliberation. According to Mauro Barisione, the framing of the setting includes:

• Who is promoting the event and who the sponsors are: Whether the public trusts the promoters/sponsors and the feelings they associate with them (including any explicit views the sponsors promote) may impact the framing of the topic at hand.
• Where is/are the deliberative session(s) being held “(i.e., institutional, academic, civil society or business organization, etc.)”: Apart from accessibility concerns, the location of the deliberative session(s) sends implicit messages to participants who may have previously had negative experiences at governmental institutions, for instance.
• Who are the witnesses and experts brought on as part of the project: This point is most obvious. Especially when it comes to less well-known topics, witnesses and experts may be the first window into a new topic for participants. A biased selection of experts (or including experts who provide unscientific evidence) can significantly impact deliberation. Moreover, an oft-neglected point worth considering is the diversity of expert witnesses who present relevant information at public deliberation events. Selecting experts is discussed in more detail in a later section, but it’s worth noting here that a diverse group of expert presenters will ensure a diversity of views and may aid in building trust with participants as well. 

Selecting a Type of Public Deliberation

Another factor that merits attention at this point is the type of public deliberation being undertaken. Though public deliberation has been referred to as one entity thus far, there are many different types, including, but not limited to, citizens’ juries, planning cells, consensus conferences, citizens’ assemblies, and deliberative polls. Below are some further details about various types of public deliberation (where a source is not included below, it was adapted from Smith & Setälä).

Citizens’ juries

• Description: “In common with the legal jury, citizens’ jury assumes that a small group of ordinary people, without special training, is willing and able to take important decisions in the public interest” (Coote & Mattinson)
  • Selected jurists are meant to represent a microcosm of their community (Crosby)
  • Jurors are selected through a quota process that often takes into account demographics (e.g., age, gender, education and race) or jurors’ prior opinions about the question at hand (Crosby)
  • Neutral moderator moderates discussions (Crosby)
  • Jurors are able to determine what questions are asked of the witnesses
  • Similar to planning cells (below), jurors are paid for the time (e.g., $75 per day)
• Number of participants: 12–24
• Time: 2-4 days (Armour)
• Output: Jurors put together a written report of their conclusions (Armour)
• Examples: Ned Crosby of the independent Jefferson Institute in the US initially ran them with groups of 12-24 people (Smith & Wales)
  • Another example is the Oregon Citizens’ Initiative Review

Planning cells

• Description: Deliberation includes 3 phases (Participedia):
  • Citizens are presented with pertinent information from multiple perspectives and ask clarifying questions
    • Often two cells are run in parallel, one starting an hour later than the other so that expert witnesses can present information twice within one visit (Dienel)
    • Two trained moderators are required for each planning cell (Dienel)
  • Cells are broken up into smaller groups of 5 and work together to come up with recommendations regarding the topic
  • The full cell reconvenes and each mini group presents their top recommendation, which is then assessed by all members of the group, until some final recommendations are made
• Number of participants: 25 in each cell (multiple cells running simultaneously or one after another); often have 6-10 planning cells total (Dienel)
• Time: 3-4 days (Dienel)
  • Each day is split into four chunks of time, each spent on a different “thematic focus”
  • The final day is used to summarize the participants’ thoughts and come to conclusions (Planning Cells Database)
• Output: Citizens’ report
  • Created by the planning committee based on quantitative data aggregated from all the cells (for instance, from how participants responded to which option they favored) (Dienel)
  • The first draft of the report may also be discussed with participants in a follow-up meeting
• Example: Cologne Town Square

Consensus conferences/citizens’ conferences

• Description: These conferences have been described as a citizens’ jury plus a town hall meeting (Einsiedel & Eastlick)
  • Citizens learn some basic facts about the issue at hand and formulate questions they’d like addressed (Kenyon). In other words, the topic is chosen by the organizers while the concrete problems are selected by the citizens (Nielsen et al.).
  • Witnesses are called to answer these questions (Kenyon) and the experts are selected entirely by citizens (Nielsen et al.)
  • A steering group is often involved who ensures that “the conference is balanced and just” and provides support to the organizers (Nielsen et al.)
    • Members include “scientists, representatives of non-governmental organizations, policy-makers and others, who are engaged and informed concerning the topic”
  • It’s recommended that fact sheets be prepared that explain basic definitions necessary for understanding the topic as well as pros and cons for all sides — this will be provided to participants in preparation for discussions and the questioning of expert witnesses (Nielsen et al.)
    • Introductory materials should also clearly explain the role of the consensus conference participants and how their contributions will be used to inform future decisions
• Number of participants: 10-24
• Time: 4 days
• Output: Report written by citizen participants (Kenyon)
• Example: Danish Board of Technology consensus conferences

Citizens’ assemblies

• Description: Typically larger and longer than the deliberative processes discussed above, similar to legislatures composed of regular citizens (Lacelle-Webster & Warren)
  • Organizers often aim to make assemblies as representative of the public as possible through methods like stratified random sampling (Lacelle-Webster & Warren)
  • More limited participation opportunities so some suggest that it’s better to call citizens’ assemblies “representative institutions” rather than a form of participatory democracy (Lacelle-Webster & Warren)
  • Participants compensated for their time (Ferejohn)
  • Based on the initial citizens’ assembly in British Columbia, any interested expert witnesses or groups were able to testify in front of the participants (Ferejohn)
  • Participants can ask any questions of the witnesses (Ferejohn)
  • A Chair — who was the only public official directly involved in the process — is able to make obligatory rulings as to how the deliberations would proceed
  • Organizers set the agenda
  • Formats can vary: the inaugural 2004 British Columbia citizens’ assembly was more so in the format of a legislature as opposed to an Irish citizens’ assembly which included roundtable discussions with groups of 7-8 participants alongside a facilitator and notetaker at each table (Farrell et al.)
• Number of participants: 99-150
• Time: Meetings over several months (often during weekends) (Farrell et al.)
• Output: Report and recommendations from citizens’ assembly (Ferejohn)
• Example: The 2004 British Columbia Citizens’ Assembly

Deliberative polls

• Description: Combines traditional polling with the benefits of participatory processes
  • Randomly selected participants are polled on their opinions about a topic (Stanford Deliberative Democracy Lab)
  • Participants receive introductory materials about the issue prior to the event (Stanford Deliberative Democracy Lab)
  • A weekend-long event where participants learn more about the issues from experts (ask questions from experts) and discuss issues with one another with trained facilitators present (Stanford Deliberative Democracy Lab)
  • At the end, complete the questionnaire from earlier once more (Stanford Deliberative Democracy Lab)
  • Participants often paid for their time ($75-$200) (Participedia)
• Number of participants: 200+
• Time: One weekend (Stanford Deliberative Democracy Lab)
• Output: Differences measured between the opinions of participants pre- and post-deliberative polling process (Stanford Deliberative Democracy Lab)
• Example: 2008 deliberative poll on housing shortages in San Mateo County, California

A note on online deliberation

The COVID-19 pandemic forced many initiatives to shift to a fully online modality. This highlighted many of the opportunities as well as challenges that online deliberation presents. One consideration is accessibility, a double-edged sword when it comes to deliberation. Virtual deliberation alleviates the need for a venue or hotel accommodations — decreasing costs for organizers — and may allow participants to continue to go to work at the same time. However, difficulties with using technology and a lack of access to a device or an internet connection are drawbacks. Another opportunity presented by virtual deliberation is to provide more balanced viewpoints on the topic of deliberation. For instance, there are no geographical barriers as to the experts organizers can invite to speak at an event. 

A concern somewhat unique to online deliberation is data privacy and security. While this can also be an issue with in-person initiatives, many tools that participants are familiar with and may prefer to use do not have robust security.

A note on cost

While the cost of many deliberation initiatives is not publicly available, the available estimates range from $20,000 (citizens’ jury) to $95,000 (consensus conference) to $2.6 million (Europe-wide deliberative poll of 4300 people) to $5.5 million (citizens’ assembly). Note that these costs come from a range of time points and locations (though they have been adjusted for inflation) and only serve as rough estimates. A major contributor to these costs, particularly for longer deliberative initiatives, is hotel or venue costs as well as the reimbursement of participants. This reimbursement is costly but a part of the founding philosophy of many types of deliberation, including that of planning cells.

Selecting Participants

Many different approaches can be taken to selecting participants for deliberative forums. Unfortunately, there are inherent trade-offs in selecting a sampling method or approach. For instance, random sampling is more in line with the principle of “equal opportunity” and may promote “cognitive diversity”— the diversity of ideas, experiences, and approaches participants bring to the event — but is prone to creating deliberation groups that are not representative of the population at large. This is particularly true when the deliberative forum has few participants. This is why, depending on the type of deliberation event (and therefore number of participants chosen), a different type of sampling may be appropriate. 

Another approach is random-stratified sampling, where participants are randomly chosen and invited to participate in the deliberative event. There is often an unequal distribution among those who accept the invitation — for instance, individuals with higher socio-economic statuses may respond disproportionately more. In this case, a more representative sample may be chosen from those who responded. Quotas may also be set, such as ensuring that a certain number of female-identifying participants are included in a deliberative event. For this method, the organizers must decide on groups of individuals who are primarily affected by the topic being discussed, as well as groups often excluded from such deliberations. A deliberative forum on immigration, for instance, may call for the presence of a participant who is an immigrant to ensure polarization does not take place. In certain instances, purposive sampling — where individuals from groups whose views are specifically being sought are purposefully chosen — may also be appropriate. Furthermore, some researchers suggest including a “critical mass” of individuals from typically underserved groups. This can serve to make participants more comfortable in speaking up, ensure that the diversity of discussions is retained when participants are broken up into smaller groups (in certain forms of public deliberation), and provide a step in avoiding tokenism.

Furthermore, there are newer methods of selecting participants that combine both random and stratified sampling — namely algorithms that try to maximize both representation and equal opportunity of participation. One instance is the LEXIMIN algorithm which “choose[s] representative panels while selecting individuals with probabilities as close to equal as mathematically possible.” This algorithm is open-access and can be used at panelot.org. 

Aside from considerations for selecting participants, it’s important to consider the selected individuals’ ability and willingness to participate. Several factors can dissuade selected individuals from taking part, including but not limited to, the cost of missing work, the cost of childcare, transportation costs, and lack of trust in the organizing body or agency. Prohibitive costs are addressed by several of the deliberation models discussed in the “Selecting a Type of Public Deliberation” section. These models strongly suggest stipends which, at minimum, cover incidental expenses. A lack of trust is a particularly important issue to address as it can hinder the organizer’s ability to reach individuals typically left out of policymaking discussions. One approach to addressing this once again brings us to making — and critically, keeping — promises regarding the implementation of the conclusions of participants. Framing (as discussed in an earlier section) can also contribute to building trust, though, importantly, this is not a gap that can be bridged overnight. A more extensive discussion on inclusion in public deliberation forums can be found here.

Bringing On Experts & Creating Materials

Prior to selecting the group who will participate in the public deliberation activity, steps need to be taken to organize which experts will be part of the event and create the informational material that will be provided to participants before deliberations begin. 

Here, efforts must be made to ensure sufficient and balanced information is presented without creating a framing event where participants enter discussions with a biased perspective. It has been found that participants readily integrate the facts and opinions presented by experts/witnesses prior to deliberation and critically engage with their points. A deliberative engagement initiative in British Columbia, Canada about biobanking brought on a variety of experts and stakeholders to present to participants. To ensure fairness, presenters were “given specific topics, limited presentation times, and asked to use terms as defined in the information booklet” that was previously provided. A unique component included in this initiative was the ability for participants to ask presenters questions in between the two deliberative session weekends, which were two weeks apart, through a website. 

In addition, participants were provided with booklets and readings. In the case of the British Columbia initiative, to create booklets and background materials, a literature review was performed. Once more, the materials should provide a balance of opinions. They should include the most important facts relevant to the question at hand, some of the most common/salient approaches and points with regards to the question, and the weaknesses of each approach/point (Mauro Barisione). It is also best to keep materials succinct, with some deliberative initiatives keeping their materials to one page long.

Though the traditional approach is to have experts present prior to deliberation, other methods have also been used. For instance, a Colorado deliberation initiative focused on future water supply used an “on tap but not on top” expert approach. Rather than call experts to present information, they instead provided one-page information sheets, followed directly by deliberation. Experts were present during the deliberation session. When prompted by a participant, a facilitator would ask an expert to briefly join the group to answer the participant’s question. The approach was largely successful, though one “rogue expert” frequently interjected in a group’s discussion, providing his own opinions. One limiting factor to this approach is time; the deliberative sessions mentioned above were two hours long. But many other forms of deliberation are significantly longer, making coordinating with experts for long durations of time difficult. Despite these challenges, this approach provides an interesting way of integrating experts into the deliberation process so their expertise is best used and the participants’ questions are best answered as they arise.

Facilitation

A good facilitator or moderator is critical to the deliberation process. As explained by Kara N. Dillard, moderators set the ground rules for the discussion and prevent any one participant from dominating the session; this is called presentation. It has been found that clearly setting expectations for the discussion can lead to greater deliberative functioning — which, for our purposes, includes the exchange of ideas/reasons, equality, and freedom to speak and be heard — according to participants. Moderators also guide the discussion in two main ways: asking questions that challenge what participants have already discussed (elicitation); and connecting ideas that were previously brought up to new topics and “play[ing] devil’s advocate” to bring forth new ideas (interpretation). At the end of the session, moderators also help participants produce conclusions by asking what areas of consensus and contention were present throughout the discussion.

Moderators can take multiple approaches to facilitating, with one framework proposed by Kara N. Dillard separating moderators into three groups: passive, moderate, and involved. Passive moderators take a “backseat” approach to moderating. They often describe their role to participants as only being there to prevent a participant from dominating the conversation, rather than actively leading it. This has led to unfocused discussions and unclear conclusions. Participants often jumped around and went off-topic. Though this passive approach may work in some instances, a moderate or involved approach often leads to better deliberation.

Involved facilitators actively lead the discussion by asking questions that challenge participants to think in new ways, sometimes acting as a “quasi-participant.” In line with this, these moderators often play devil’s advocate to move the discussion in new, albeit related, directions. These moderators ask follow-up questions and “editorialize” to help participants flesh out their ideas together and aim to pinpoint points of contention so participants can further discuss them. If participants begin to veer off-topic, involved moderators will move the group back into a more focused direction while also connecting this new topic to the main question, allowing for new thoughts to emerge. These moderators take the time to sum up the main points brought up by participants after each point so conclusions become clear. Once more, this approach may not work in all instances but often leads to deeper conversations and more focused conclusions.

As implied by the name, moderate facilitators are somewhere in between passive and involved facilitators. These moderators ask questions to guide the discussion, but don’t often challenge the participants and let them take the wheel. These moderators use the elicitation strategy frequently, an important difference between moderate and passive moderators.

Due to the skills needed to facilitate a deliberation event well, organizers or government agencies looking to organize these events may require would-be facilitators to undergo brief training. 

What Comes Next

After deliberation has taken place, the next step is to write a report summarizing the conclusions of the deliberative forum. As we have seen several times with other topics, there are multiple approaches to this. One approach is to leave the report writing to the facilitators, organizers, or researchers who use their own takeaways from the deliberation (in the case of facilitators) or summarize based on recordings or transcripts (in the case of organizers or researchers). However, this method introduces bias into the process and doesn’t allow participants to be directly involved in creating conclusions or next steps.

An alternative is to allocate time towards coming up with conclusions together with participants both throughout and at the end of the deliberative session. Recall that involved facilitators frequently summarize the conclusions of the group throughout the deliberation, making this final task both more efficient and more participant-led. Participants can directly and immediately add on to or push back against the facilitator’s summary. As a guideline, Public Agenda, an organization conducting public engagement research, divides the summary into the following sections: areas of agreement, areas of disagreement, questions requiring further research, and high-priority action steps.