Deliberative Polling for Constitutional Change in Mongolia: An Unprecedented Experiment

With a view to ensure the effective representation of public views, the Mongolian government employed Deliberative Polling in the constitution reform process. Despite some challenges and criticism, the process allowed a random sample of ordinary citizens to participate in and influence the agenda and outcome, including the rejection of crucial proposals supported by the main political parties. Professor Fishkin and MP Zandanshatar – two insiders of the process - conclude that the experience attests to the unique value of Deliberative Polling as a modality of public participation in constitutional reform processes.

For the first time ever, the Mongolian central government has officially convened a random sample of its entire citizenry to deliberate about possible amendments to its constitution. The results have gone to the Parliament as the first step required by law in possibly amending the constitution. All over the world there is a great deal of academic discussion of ‘deliberative democracy’. Mongolia has taken a major new step in making it practical for constitutional issues.

The project was conducted in accordance with the new ‘Law on Deliberative Polling’ which specifies the requirements for organizing this kind of consultation. Deliberative Polling is now required by the law before the Parliament can proceed to consider amendment to few crucial constitutional provisions affecting parliament, the presidency and the prime minister. Two working groups with cross party membership worked on the initial agenda after extensive public consultations and research on possible amendments. An Advisory Group of distinguished civic leaders had final authority over the process and its recommendations to Parliament. All the scientific research was conducted by the National Statistical Office of Mongolia (NSO) with technical advice and support from Stanford University’s Center for Deliberative Democracy.

Deliberative Polling is now required by the law before the Parliament can proceed to consider amendments to crucial provisions.

The basic idea of Deliberative Polling is simple. A Deliberative Poll surveys a random sample of the population, both before and after it has been brought together to deliberate about the issues in depth. The success of the process requires a representative sample, an appropriately designed questionnaire administered both before and after the deliberations, balanced briefing materials as a basis for discussion, moderators for the small group discussions who are trained not to offer any hint of their own views, and questions from the small groups directed at competing experts in plenary sessions. The idea is to engage a representative sample with the best practical conditions for the people to really think in depth about the issues. Results are reported in the before and after questionnaire results, often showing significant changes of opinion. In this case, the quantitative results were the basis for recommendations to the Parliament about which proposals had strong enough support to merit consideration in a constitutional amendment.

A Representative Sample of the Nation Gathers in Ulaanbataar

The NSO randomly selected households from randomly selected geographical areas (or strata) and then randomly selected an adult in each of those households to be interviewed. In effect, each adult citizen in the country had an equal random chance of being selected. This approach is widely recognized around the world as one of the ‘gold standard’ methods for scientific sampling. However, like all methods of sampling it can be undermined if the response rate is low. In this case, the response rates were exceptionally high so we can have a very high level of confidence in these results.

Each adult citizen in the country had an equal random chance of being selected.

So as to cover the entire country, 1 568 households were randomly selected within geographical areas. A strict scientific process was followed to randomly select people within the selected households. Tablets with GPS kept track of the interviewers and their progress in completing the fieldwork. Of the total number of people selected (1 568), 96 per cent (1 515) completed the initial interview. In conformity with the law on Deliberative Polling, more than half of those who completed the interview were invited. Out of the 785 who were invited, 85 per cent (669) came to Ulaanbaatar and completed the entire weekend of deliberations (on 29 and 30 April 2017). This is an extraordinarily high rate of participation. The NSO Report (English version here) has more data on representativeness.

What Happened?

It is worth noting the opinion changes as well as the top rated proposals. Support for two of the most ambitious proposals dropped dramatically with deliberation. The proposal for ‘Creating a Parliament with two chambers: a people's representative body (People's Great Khural) and legislative body (State Baga Khural)’ went from 61 per cent to 30 per cent, a drop of 31 points. With deliberation, the participants became more skeptical that ‘a second chamber would provide effective oversight of the lower house of Parliament’. Agreement with this idea dropped from 70 per cent to 38 per cent. More specifically, there was increased agreement with the criticism that ‘both chambers would be controlled by the same political parties, thereby not providing proper oversight’. Those agreeing with this proposition rose from 43 per cent to 59 per cent. There was also a significant increase from 48 per cent to 57 per cent in those who agreed that ‘adding a second chamber would create too many politicians’.

Nine of the top ten proposals at the end of the deliberation concerned the transparency, accountability and meritocratic operation of government through protections for the civil service and the judiciary.

A second major drop in support occurred with the proposal for an indirectly rather than directly elected President for only a single six year term. There were two components to this proposal, the change in the term and indirect election. Support for ‘Electing the President for a single six-year term, without reelection’ dropped from 61.5 per cent to 41 per cent with deliberation. Support for ‘Electing the president for a single six year term by an expanded plenary session of the Parliament that includes Parliament members and the Citizen’s Representative Councils of aimags (provinces) and the capital city’ started at 36 per cent and ended at 33 per cent (not a significant drop but showing a low level of support after deliberation). There was a significant drop in support for one of the arguments in favor of indirect election:  ‘If the President is indirectly elected by the Parliament and the Citizens Representative Councils, then he/she will be someone acceptable to all sides and above political fray’. Agreement with this conclusion dropped from 55 per cent to 38 per cent. On the other hand, there was strong agreement before and after with one of the key arguments in favor of direct rather than indirect election: ‘If the President is directly elected s/he can better speak for the interests of all people’ (84 per cent before, 80 per cent afterward, no significant change).

By contrast, the deliberators supported an amendment that would increase the power of the Prime Minister: ‘Granting the Prime Minister the authority to appoint and dismiss the members of his/her Cabinet’. This proposal increased significantly from 57 per cent to 73 per cent.

Many of the highest ranked proposals after deliberation were also rated highly beforehand. All the proposals were evaluated on the same scale from 0 to 10 where ’0 means strongly oppose and 10 means strongly support and 5 is exactly in the middle’. Nine of the top ten proposals at the end of the deliberation concerned the transparency, accountability and meritocratic operation of government through protections for the civil service and the judiciary (the tenth was granting authority to the prime minister to appoint and fire members of the cabinet).

Each of the two proposals that showed a large drop in support, the proposal for indirectly electing the President and the proposal for a second chamber of Parliament, was supported mostly by members of one of the two major parties.

The quantitative results before and after deliberation, complied by the NSO, were given to the Advisory Committee which reported them to the Parliament. Each of the two proposals that showed a large drop in support, the proposal for indirectly electing the President and the proposal for a second chamber of Parliament, was supported mostly by members of one of the two major parties (the indirect election of the President was mostly supported by the ruling Mongolian People’s Party and the second chamber was most prominently supported by members of the main opposition Democratic Party). The Deliberative Poll appears to have screened out both proposals from further consideration. The Parliament will soon reconvene and consider an amendment which in draft form has the proposal for the increased power of the prime minister and a number of provisions for strengthening the independence of the civil service and providing accountability. It does not include either the indirect election of the president or the second chamber. The Parliament will continue to engage in other public consultations and deliberations before deciding what to do. But it is already clear that this scientific national public consultation has had a major impact.

The recent election of a new President provides an additional source of uncertainty, especially since some of the issues affect the powers of the President.

The Parliament can amend the constitution with a 3/4 majority and the current ruling party, the Mongolian People’s Party, has a larger majority than that. Hence, it could, in theory, proceed to amend or it could take the issue to a referendum, or it could drop the amendment. The public consultation period is expected to end in September 2017, before Parliament decides on the next steps, in particular if a referendum would be conducted, on top of the adoption of the proposals with a 3/4 majority.  In the meantime, the candidate of the ruling party lost in the July 2017 presidential elections, and the Prime Minister subsequently lost a no confidence vote in the Mongolian Parliament. While the implications to the ongoing constitutional reform remain unclear, given the focus on addressing corruption and empowering the office of the prime minister, and the continued dominance of the ruling party, the process is likely to continue. However, the recent election of a new President provides an additional source of uncertainty, especially since some of the issues affect the powers of the President.

A Critique

Professor Munkshaikhan Odonkhuu published a critique of the process in ConstitutionNet, noting some ‘leading questions’ in the Deliberative Poll and ‘unbalanced information’ in the briefing material in relation to some of the themes. Nevertheless, two of the questions he quotes do not exist in the questionnaire, and one is quoted without reference to the stem of the question (which renders it in a balanced format). The questionnaire, the briefing document and all the results are available online both in Mongolian and in English. While he might believe the questions he quotes are leading questions, all the policy proposal questions in the Deliberative Poll follow a precise format for balance. There were two formats. The first format is:

‘Now here are some proposals we would like your opinions on. Not everyone has thought much about these. But we do need to ask them. If you come to one you haven’t thought about, just say so, and we’ll move on to the next one. On a scale 0 to 10 scale, where 0 means strongly oppose, 10 means strongly support and 5 is exactly in the middle, how strongly would you support or oppose the following changes to the Constitution?’

Each proposal was rated on this 0 to 10 scale.

The second format put responses on a 1 to 7 scale. Here is an example:

Some people think if the President can run for re-election, that makes him more political. Suppose these people are at 1, on a 1 to 7 scale. Other people think that if the President can run for re-election that makes him more accountable.

Suppose these people are at 7. Where would you place your views on this scale?

This format, which comes from the American National Election study, is also a widely tested and accepted format for questions. Under accepted standards of public opinion research, we believe that neither format could be considered unbalanced or as constituting ‘leading questions’. Why is this apparently technical issue important? Because if the Deliberative Poll questions were ‘flawed’, that might imply that the quantitative results were skewed by measurement error. But since two of his questions are not actually in the questionnaire and the third is balanced when quoted in full, this argument is irrelevant.

All the policy proposal questions in the Deliberative Poll follow a precise format for balance.

The second major argument in the same piece is that the briefing document is unbalanced and was not properly distributed to the participants:

‘A briefing booklet distributed to the participants was not available until the day of the second polling and did not include balanced information. For instance arguments for establishing a new constitutional organization called the State Control Organization were covered in five pages yet all other issues were given only two pages.’ (emphasis added).

In fact, the briefing booklets were distributed to virtually all the participants either before the weekend or on arrival. The discussions made constant reference to the materials and as we walked around the small groups we saw the participants employing them by turning to the summary pages for the proposals being discussed. Furthermore, the briefing book had a total of 59 pages, not a total of seven pages as the quote above might imply. Most importantly, each of the 18 proposals had a page with a tabular format where the proposal was clearly spelled out, and key arguments in favor and arguments against were listed side by side. The moderators were trained to draw the participant’s attention to this page when each proposal was discussed.

Evaluations from the Participants

At the end of the weekend, the deliberators were asked to evaluate all the main components of the process. Their evaluations strongly support the balance and substantive character of the proceedings. Overall, 89.3% said ‘the event as a whole’ was valuable, 76.1 per cent thought the small group discussions were valuable, 79.4 per cent thought the briefing materials were valuable, and 82.5 per cent thought the plenary sessions with competing experts were valuable.

On more specific aspects, 95.8% agreed that ‘my group moderator provided the opportunity for everyone to participate in the discussion’ and 92.5% agreed that ‘the members of my group participated relatively equally’. Similarly, 67.8 per cent agreed that ‘my group moderator tried to make sure that opposing arguments were considered’ and 93.8 per cent agreed that ‘the important aspects of the issues were covered in the group discussions’. Only 17.2 per cent thought the group moderator ‘sometimes tried to influence the group with his or her own views’. In reflecting on the event as a whole, 88.2 per cent agreed with the conclusion that ‘I learned a lot about people very different from me—about what they and their lives are like’. Given that the process engaged a national random sample randomly assigned to small groups, the diversity the participants encountered would have reflected that of the entire society.

The deliberative process represented what the entire country would think—if it were thinking in depth and had a chance to get its questions answered. 

Considering the importance of this unique national experiment in applying deliberative democracy to constitutional change, we believe the process and its results deserve to be considered on their merits. Instead of focus groups, self-selected meetings, and conventional polls where the public is not informed, this process represented what the entire country would think—if it were thinking in depth and had a chance to get its questions answered. What better subject for such a process than whether the constitution of the nation needs to be amended?

James Fishkin directs the Center for Deliberative Democracy at Stanford University; Gombojav Zandanshatar is Member of Parliament in Mongolia.