Transcript, Meeting 21, Session 1


May 27, 2015


Philadelphia, Pa.


James S. Fishkin, Ph.D.
Professor and Director of the Center for Deliberative Democracy
Stanford University

Scott Kim, M.D., Ph.D.
Senior Investigator
NIH Department of  Bioethics

Download the Transcript



DR. WAGNER:  So I ask Drs. Fishkin and Kim, please, to join us at the table here. We're going to dive into our work on deliberation and bioethics education.  And this panel is focused on "Connecting Democratic Deliberation Theory to Practice."     

In our usual format, we will ask, gentlemen, to hear from each of you.  And we will hold questions until both of you have had a chance to make your presentations.     

And so we will begin with Dr. James Fishkin, who is the Janet M. Peck Chair in international communication, professor of communication, professor of political science by courtesy and Director of the Center for Deliberative Democracy at Stanford University. Dr. Fishkin is the author of a number of books, including "When People Speak:  Deliberative Democracy and Public Consultation."  And he is known for developing deliberative polling, which is a practice of public consultation that employs random samples of the citizenry to explore representative and informed opinions on public policy topics.     

Dr. Fishkin has been a Guggenheim fellow, a fellow of the Center for Advanced Study in the Behavioral Sciences at Stanford and a fellow of American Academy of Arts & Sciences. Welcome.  It's good to have you with us.     

DR. FISHKIN:  Thank you so much. It's a great pleasure to be here.  Very nice to see Amy again and to meet the rest of you.  And to have experienced that moving window into the work of the Commission in the last few minutes, it was really a privilege to hear it.     

So I want to talk -- I've only got ten minutes, so I have a bunch of slides.  But let me just talk and then if I get to any of the slides, so much the better.     

DR. GUTMANN:  Or not.     

DR. FISHKIN:  Yes, right.     

Well, so the basic idea.  So I became interested some years back and it took over my life, this problem of how to consult the public in a representative and thoughtful way.  So, actually, the book is "When the People Speak."  So it's not just when people speak, but it's a gathering of the people in this kind of representative and thoughtful way.     

And I think the best way to do that is with random sampling because there are two -- there are two questions about any deliberation. Who is deliberating?  I mean, this is a very impressive, elite or expert deliberation.  But there's some questions for which it seems appropriate to consult the people.  And then if you consult the people, there's the question of which people.  Because very often the simplest thing is just to ask people.  And then you get the people who feel strongly.  They turn up.  They speak loudly.  They may not even be prepared to listen to others.  They may not really be prepared to deliberate.  So you need an institutional design to consult the people.     

The institutional design that we have come to, I thought I invented, but in fact it really goes back to ancient Athens.  In fact, I have a slide of the kleroterion, the machine that the Athenians used for random sampling.  I photographed it in the Museum of the Forum.  So it still exists, this machine.     

In any case, the basic idea of random sampling -- so I combine random sampling and good conditions for things -- good conditions for deliberating are not so different from what Amy and Dennis have talked about and other theorists have talked about, but we try to make it practical.  So what we do with the so-called deliberative poll is -- and we've done about 70 of these in now 22 countries, the most recent in Tanzania, which was on NPR last week.     

The -- we -- we first -- first, there's an issue.  It's got to be about something. We have an advisory group.  The advisory group considers policy options that might be ripe for decision.  It's got to be a balanced advisory group representing different points of view.  Sometimes we also do focus groups and things earlier to sort of suggest the way the people think about an issue.     

But anyway, we develop briefing materials on the issue.  We've been doing a whole series in Africa, believe it or not.  And the African cases we've had to have video versions of the briefing document, oral interviews, because the people can't fill out self-completion questionnaires.  Otherwise it's worked perfectly in Africa just as in every place else.     

Everywhere around the world, the people -- what we found is the people are smart if you create the conditions for collective deliberation.  But, normally, people -- I've got colleagues in political science who have made a good living out of saying the public is stupid. The public is not stupid.  It's just, normally, they're not paying attention.  So if you can give them a reason where they think their voice matters, they do very well.     

So to step back, we have the briefing materials.  We have a good questionnaire not only about policy options, but pros and cons and reasons for adopting one policy or another and all kinds of explanatory variables.  And then we get as good a random sample as we can in an initial survey.     

And then we recruit the people to come.  Amazingly, in Africa, almost everybody who took the initial survey came.  In Tanzania, I think there were close to 400 people who took the initial survey and almost everybody came for a national project three or four weeks ago when I was there.     

Well -- and we're doing one in China this weekend, in Shanghai, on different topics. Any -- but the topics are complicated, they're tradeoffs, they involve policy tradeoffs, they involve different perspectives in reasoning. Sometimes they are topics that are very controversial.     

We've even done -- I'm writing a -- we're writing a paper now about the project we did in Bulgaria on the condition of the Roma because it was part of the process that led to the desegregation of the schools for the Roma.  And, also, we did the -- in Northern Ireland between Protestants and Catholics.  So these are cases that are very contested.     

Anyway, the basic idea is we bring the sample together.  We have a survey of the people who didn't come so we can evaluate the representativeness.  We engage in one, two, three days of deliberation in small -- alternating small groups and plenary sessions with trained moderators.  And then in the plenary sessions, answer questions from the small groups.  Then at the end, they take the same survey as before with some additional questions.  And we see the changes of opinion.  We think maybe about 70 percent of the policy attitude questions we've ever asked have changed significantly with deliberation.     

And we also can demonstrate systematically that the things that go wrong with other designs, like Sunstein's polarization or the domination by the more educated or the more advantaged or the men, none of those things happen in the deliberative poll.  We've demonstrated that ranging over lots of different projects.     

And so the reason is we have conditions where people can really engage with each other, they listen to each other, they -- they become more informed.  We have information questions so we can measure that.     

And the results provide what I call a route to responsible advocacy.  How the people change and why.  And then that can be invoked for policy choice.  It can also be expanded up; that is, you can radiate it with the media.    

We did a big project in California a couple of years ago with the NewsHour that -- about how to fix the state, which appeared to be broken at the time.  And we broadcast it with -- Judy Woodruff did a documentary about it -- but we broadcast which policy choices the people would support and which they would not and why.  And some of them have led to some actual -- one of them was -- led to some legislation or was part of some legislation.     

It's too difficult to capsulize everything.  I will say we've done a few projects related to medical ethics, not many, with Dan Wickler and Dan Brock some years ago.  I got involved in a deliberative -- a series of four deliberative polls in Thailand about renal replacement therapy, whether it should be incorporated into what was -- what used to be called the 30 bot program.  And a version of it was -- and the public's reasoning was very interesting in the way that -- I think the decisive point was that the Government employees had renal replacement therapy and the rest of the public didn't.  And then if they had a kidney disease, their entire kinship network would be devastated as people would try to raise the money for it.  So that was one such case.  But then there was a coup and various things in Thailand.  We haven't -- with the collaborators, we never published it, but we did these four projects there.     

We also did -- the 50th anniversary of the National Health Service in Britain we did with Channel 4.  And issues about rationing or postal code rationing and various other aspects of the health service.  So we've done these, but the point is the method, the approach of making deliberative democracy practical, and if you don't -- if you don't think about the sample, you're going to get -- if the sample is too small, as with a citizen jury, it's not representative. Statistically, it's meaningless.  If you don't collect the opinions and confidential questionnaires, you get the social pressure of a jury, social pressures for consensus.  And if you have self-selection, then you're swamped by the people who feel strongly to show up.     

So if you have a really good representative sample in attitudes as well as demographics and they deliberate in depth about the issues, you will get results that are worth implementing and that very often have been -- even though they're usually not required to implement them, but the sponsorships can be set up in such a way that there's a successful route to implementation.  And we're -- I'm writing a book now about that.     

And so I think that deliberative -- also, we've experimented in the classroom.  The same materials and the same process can be replicated in the classroom and used as a form of public education.     

So it can also be done online. We've done it online.  In fact, Google Hangouts are free, and so we did a project on immigration reform with Google Hangouts.  As a controlled experiment, it worked extremely well with very interesting results.    

So -- does that mean I've run out of time?     

DR. WAGNER:  We need you to wrap up, yes, sir.     

DR. FISHKIN:  Yes.  Okay.  Sorry. I've rambled a bit.  I didn't get to show any of my slides.  The slides range across Japan and China and Africa.  And, actually, the public health cases in Africa touch on bioethics because they all touch on disease, so maybe I'll come back to that in the questions.  Thank you.     

DR. WAGNER:  Very good.  Thank you. Thank you.     

Let's move on to Dr. Scott Kim.  He is a senior investigator in the Department of  Bioethics at the NIH clinical center.  Previously, he was professor of psychiatry and co-director of the Center for Bioethics and Behavioral Sciences and Medicine at the University of Michigan.     

Dr. Kim uses normative analysis, experimental surveys, decision analysis, democratic deliberative methods to address a variety of bioethical issues, including assessment of decision-making capacity, research involving the decisionally impaired, surrogate consent for incapacitated patients and prevention trials for Alzheimer's disease.  Quite a spread of interests.     

Thank you for being here, Dr. Kim.     

DR. KIM:  Thank you.  And thank you especially for this opportunity to discuss some of my research.  I'm particularly honored to be here today at the same table with Professors Gutmann and Fishkin because really -- I will present a case study, an application of the theory, as well as the practice that Professor Fishkin has laid out over, you know, just many studies that have been really pioneering work in this field.  And I would say that the method, the intent of how we approach the application of democratic deliberation for bioethics has been inspired by the framework that Professor Fishkin has provided.  So I am going to use the slides and really run through them rather quickly.  So I have to say this.     

So I will just use this as an opportunity to discuss how the various types of studies, basically, that Professor Fishkin has mentioned could be applied to bioethics and I'm going to apply it for the case of surrogate decision making for dementia research.     

We've used this in several topics now and we have one ongoing topic.  For the lack of a really elegant term, I've used the term "normative opinion survey," meaning simply that we accept the premise that the public's value should be incorporated into public policy, and not just as a strategic marker, but rather as something that holds normative weight.     

And we use different terms in theory, "equal deliberation," "equality of voices" and so forth.  And that's what bases it.  But how do you measure this?  This is quite a task.  And you've already heard eloquently how challenging that can be.     

Now, there are two aspects, the information challenges, which have to do with the complex scientific, legal and historical and ethical dimensions, as well as the need for unbiased, accurate and reasonably comprehensive information.  And we all know we could accomplish this by doing quick telephone surveys.  No, actually not.  So that's the challenge, right?  And even if you can communicate the information, how do you know that you're gathering valid, informed, deliberative, considered opinions?  And that's the challenge.     

So we felt that the surrogate consent for dementia research topic was particularly good because it's an unsettled policy area, which I don't need to tell this group, with very large public health implications.  It has complex background and it's not an issue that's been widely publicly debated such as assisted suicide, for example, we've had lots of debate in the public so that expert policymakers can gauge from public reports.  But this issue is a little bit more narrow.     

So the way we've tried to do this is to do the following -- and I'm just giving you a quick framework of how we've approached this -- We tried to pair our work with a nationally representative survey.  And part of our limitations with the budget and so forth and the nature of the work is there's always this threat of selection bias, of having people come for an entire day to spend time with us.  And I'll talk about that a little bit.     

So we find that pairing the topic with the same survey we used in the deliberative sessions with a national survey is very useful. We've tended to use experimental designs with control groups just so -- since this is a rather novel application in bioethics, we want to be sure what is the secret sauce?  Is it the deliberation? Or is it just getting information out there?  Using both quantitative and qualitative outcomes.     

And the last point is that this -- it's a painstaking work to do analysis to assess the quality of deliberations because you have to go back and read the transcripts very carefully.  And we've done that.     

So brief overview of the study. Basically, a simplistic way of thinking about it is that we use four research scenarios and ask people what do you think about families just having the authority to give permission to enroll their loved ones in research when they can't do it themselves. So those are the four scenarios.     

We did a random sample within the area that we live in.  And we had three arms:  A control group, which were just doing surveys; education group, where people received information materials only; and then DD group, who got the information materials but showed up on a day like this, actually, and I called this research my wedding reception research because we tend to use venues where we have banquet tables and so forth. And believe me, there's a lot of moving parts, so it's just like planning a wedding reception.     

And we surveyed people three times: A month before the day, soon after the day or at the end of the day for people who attended the session, and we do a follow-up just to see what effect that has.     

We find that both the information materials we send to people -- which, by the way, 90 percent of the people -- this is a random sample of people agreed to do this -- actually said they read most or all of the materials we sent.  And we believe that because the people who just got the written material over the mail have the similar improvement in knowledge scores as the people who show up.     

But that doesn't determine the outcome.  And I'm going to go through this busy-looking slide very briefly.  The colors -- the four colors represent the four study scenarios. And the groups represent -- groups of bars represent the groups of people.  The first set are the control group; surveys only.  Second is the people who attended the session; DD.  Third is people who got it by mail.  The two sets of graphs represent immediately after the session versus one month follow-up.     

And what you see, briefly, is this is just representing the change of opinion. There's a robust change in the middle group, as you can see, right after the session.  But surprisingly, it's retained significantly even one month after without further discussion.  There is some further inconsistent change in the information only group, but that disappears at one month.     

This is just the tip of the iceberg of the rather painstaking quality of deliberation analysis we do after these sessions.  And people, generally, on a 1 to 10 scale, they are answering in the 9 range of do they feel their opinions were respected in the group, listened to by the facilitator, do they think it's fair.  And, importantly, if you ask them the question how willing are you able to abide by the group's final position, average score is 8.8 out of 10, which is -- and there are other measures that we're not talking about today.     

So the key limitations in our study where we don't have -- well, I should say we had a generous budget from the NIH, so I shouldn't say it's small.  But it is -- it takes a lot of money and effort to do these so there is some concern, in our experience, with external validity because we were able to call and talk to about 2500 people, 2400 people, and we had over 500 people say, yes, we're going to commit to either showing up or doing the surveys.  So it was about 21 percent.     

And when we compared it to people who declined, they're younger.  And we tended to have people who had personal experience with Alzheimer's disease or dementia, people who actually had to make decisions for people with that illness or people who are related or friends with people with dementia, not surprisingly.  So we think that probably could have some impact on the outcomes.      However, that's why we think that having, first, a randomized sample of a national survey as well as a randomized design, that really pins down the internal validity of this study is really important.     

Now, we think that this selection bias could be very important for certain issues in bioethics where people's opinions may rely on trust in institutions and trust in biomedical research. And we actually are running into this in our current study of people's opinions about biobanking.  Because people who show up and want to do this -- are willing to show up versus the general public have, you know, different baseline views.  And also I mentioned that it's expensive with many moving parts.     

The conclusions we drew from our studies is that democratic deliberation has a strong effect for the topic that we discussed. Effect isn't due merely to increase in knowledge. And the process is seen as fair and trustworthy and I should add, it's like a party.  People really enjoy coming and really being listened to, being part of a discussion.  And it's almost a unique experience where people-- in a controlled setting where people -- everybody gets a chance to address an issue that's really quite complex.     

DR. GUTMANN:  If you don't mind my saying, it's the John Arras effect.     

DR. KIM:  Yes, I think that's right, very much so.     

And also our qualitative analysis generally validates the high quality of the deliberation and reasoning that we witnessed in these studies.  So I just want to thank my team and thank you for this opportunity.     

DR. WAGNER:  Dr. Kim, thank you very much.     

The floor is open for comment or question and Amy has got her finger on the button. Go for it, Amy.     

DR. GUTMANN:  I would love to.  I think these were two superb presentations and I realize how difficult it is to do ten minutes, but it gives us more opportunity to draw you out on things that would really build on what you've said that would be helpful to our report, which is going to focus on deliberation and education bioethics.     

So for both of you, how do you imagine the application to some specific bioethics issues?  Because, Jim, you said you always have some specific issues involved.  And, Scott, you represented that in your study.  So if you take -- can I just throw out the issues of vaccines -- of use of vaccines and use of antibiotics?  How would you imagine, Jim and Scott, the importance and the ability to do a deliberative exercise, deliberative polling and, Scott, your method, which is absolutely -- you know, it's consistent with Jim's. How could we, you know, advise to use that on issues like that, which are very important issues for bioethics?     

DR. FISHKIN:  Well, the process I think -- I mean, I think it would be a very suitable application because it's contested.  There are different points of view in the population. You would need -- the way we would normally proceed is you would need an advisory group that would vet materials suitable for public discussion about the -- about different policy options and tradeoffs.  You couldn't do it if everything were settled.  But if it was an open question, the idea is to consult the people on questions of collective political will.  What do they want to do and why? What are the tradeoffs?  What are the pros and cons?  Not technical questions about which vaccine is most effective, but you've got to clarify the tradeoffs for them.     

So you need an advisory group that would work -- well, this is -- the kind of group representing diverse views that would -- you've got the briefing materials.  Then you would have to decide on the population.  I think it would be very interesting to have a sample of the whole population, adult population.  And you could either do a face-to-face deliberative poll or an online -- we have now been doing online deliberative polls with video that is using -- but with small group discussions of 10 or 12 people in each small group, 300 deliberators, control group of people who don't deliberate but who get the same questionnaire before and after.  And it wouldn't be hard at all to do that.     

The question is, how to make it consequential.  You can release the results, spread it in social media.  If you have other media partners, you can disseminate it.  And if the briefing materials are really balanced and good, you can get a version of it that could be -- you can have a -- we have a classroom toolkit sort of thing that can be used in the schools or at least in colleges and universities, but sometimes in the schools, where the same process of deliberation can take place and people -- deliberation is not a spectator sport; it's a participatory sport. People can go through it and then come to their own conclusions because they're not meant to ratify a shared collective result.  They come to their own considered judgments in confidential questionnaires.  So we have done this, created a toolkit for classroom use.  And you spread it.     

And then you try to have an advisory group that is influential enough that policymakers will actually pay attention to it.  And you -- of course, this -- the kind of issue you're mentioning may vary state by state, so you might do them at the state level rather than at the national level. And so we'd be very interested in having a role or assisting in such a project if anybody wanted to do it.  It's an excellent topic.     

DR. WAGNER:  Scott, did you want to add anything to that?     

DR. KIM:  Not much.  I agree that it would be an excellent topic.  And I have no doubt that the people can handle it.  I think that one of the most important issues would be as you prepare and design this study, it has to be vetted and bought into by people who are of opposing viewpoints so that you don't spend a lot of money and effort where it's not seen as a fair -- and I think that would be the biggest challenge.  But everything else, I think there's a long history of being able to -- and you'd have to tweak things depending on the topic, but. . .     

DR. GUTMANN:  Yeah.  And where -- there are many possible topics that we've taken on, like preparedness for pubic health emergencies, how to best prepare for the next public health emergency.  That's another topic which is very open, very subject to debate.  But the worst time to have that debate is in the midst of a crisis where people's minds get frozen by the latest public pronouncement.  In the case we had in Ebola was out of, you know, the governors of New Jersey and New York on quarantine.  So this would be another -- there are many areas, but I just -- we need to think about -- you're saying we have to think carefully about which areas are ripe for this and how to do it in a way that goes into it in an unbiased way so you actually come out with a reasonable representative sampling of the people's views, which carry a lot of weight politically. And if we can't get those views or get very biased views, that's a problem for everybody.     

DR. WAGNER:  Your answers to this -- I've got Barbara and then Nelson, but I did get a question that came out of the audience that I think goes to what you folks are saying.  If you don't mind my setting it up with some observation of my own.     

One of the charges and one of the principles that this group has applied consistently is the principle of stewardship, which we have defined, in part, as being sensitive to the needs and positions of those who do not have a voice, okay, whether it's the next generation or the kinds of instances that you've been addressing, Scott. You both mentioned the critical nature of getting the right sampling.  You talked about random sampling, Jim.  You talked about nationally representative survey.     

The question is -- and maybe it's getting to the broader issue that was made about the fact that public values should be reflected in policies.  The question is:  When we have a situation that involves a small number of people or the interest of small numbers of people or even significant minorities, how does one -- how does one structure who are the people that you survey?     

Folks far more knowledgeable than I pose the untestable hypothesis that a random sample of Americans in the 1960s may not have advanced the cause of civil rights legislation in the same way that a representative democracy did; in other words, a deliberative democracy may have -- I shouldn't do it that way.  A genuine democracy may not have come to the same conclusion as a representative democracy did.     

The words from our guest, Nicholas Evans, he's in the Department of Medical Ethics here at Penn, postdoctoral fellow, he says that "Professor Fishkin distinguishes between people and the people in deliberative democracy and deliberative polling.  Given the transnational or even global nature of bioethical issues" -- so he's expanding it in the other direction -- "what criteria or justification should we adopt in defining what and who constitutes the people?"     

DR. FISHKIN:  Well, first of all, deliberative polling can -- we've applied -- we've done the entire European Union twice; that was 27 countries, the two times we did it.  Now it's 28 countries.  We've -- so you have to decide on the population you're interested in sampling from.     

Now, the issue of subgroups is very tricky.  We did a national deliberative poll in Australia before the -- about the condition of the aboriginals.  And the aboriginals are less than 1 percent of the population, depending on how you count them.     

So after a great deal of agony about this, we oversampled the aboriginals but distributed the oversample to a random half of the groups to make it a controlled experiment to see the effect of having the aboriginals in the groups. And then for certain purposes, we weighted everything back to the population when we wanted to talk about the Australian people as a whole.  But when we wanted to talk about the aboriginals, we had enough aboriginals so we could talk about them. It ended up working very well.     

We did the same thing, in effect, in Bulgaria with the Roma, although there are actually about 10 percent Roma in the population.  We had about 10 percent in the sample.  But we had enough so we could talk about the Roma -- sometimes called the gypsies -- we could talk about the Roma in the sample, but we could talk about the Bulgarian population as a whole.     

You do need enough if you want to talk about -- you need enough to generalize.  So in the European Union, we couldn't say anything about public opinion in Malta because it's so tiny as a fraction.      Now, the democratic deliberation -- yes, populism may very well lead to very different results.  Deliberative democracy is not populism. Deliberative democracy is an attempt to convene the people under conditions where they will really think about the tradeoffs and the competing values. And they have enough discussions so they can adopt the perspective of the other.  They can listen as well as speak.  And if you just have a referendum democracy or self-selected mobilization, you'll get angry voices.     

I think very often the deliberative democracy will not be far behind, and maybe even ahead, of a deliberation by policy elites.  And, of course, there's a long history of misguided deliberation by policy elites, too.  We could cherry pick examples of that, too.  But the people tend to be very thoughtful.      And one other point related to that that I want to make is sometimes when there's conflict among the policy elites and things seem intractable, the public is a lot more sensible.  We found in Northern Ireland that the people who spoke for the Protestants and the Catholics had really nearly intractable views because they had to represent their constituents and their constituents were often the people who spoke up and felt most strongly.  But when we had a random sample of the people, they had similar views, but they weren't so intractable and they were actually open to dialogue.  So the percentage of Protestants and Catholics who are willing to say the other side was actually open to reason or trustworthy went up about 16 points with the deliberation over the course of the day.  And because -- and at the beginning, they wouldn't even make eye contact with each other.  I mean, I couldn't tell who was a Protestant or a Catholic, but they knew exactly who was who.  And by the end of the day, they really understood each other.     

And it's that process of discussion, which is why I think, as Professor Kim's data also showed, just giving people information is not enough.  It's the discussion, the discussion is the magic elixir -- as you said, the secret sauce -- and it's the discussion with diverse others under conditions of mutual respect and civility.  And that's why the moderated discussion has this -- this -- this effect where people will actually put themselves in the position of those who are -- those who are affected.     

DR. WAGNER:  Got it. Dr. Kim.      

DR. KIM:  I would simply add that deliberative democracy is not just populism.  And it's based on a theory or a set of theories or a family of theories is probably more correct, as well as the design of the studies have to take into account certain normative commitments that go into how you set it up; for example, setting up in what we consider to be ideal conditions of dialogue and discussion.  We don't just let people, you know, bring their favorite polarizing arguments.  I think people have to play fair, for instance.     

DR. WAGNER:  Barbara.     

DR. ATKINSON:  I'm interested in whether the outcomes ever bring innovative kinds of things.  Or how much is it so directed by the information you give people in the first place and by who the moderator is and how the moderator moderates?  I mean, do you ever get unexpected results?  Or is it all pretty much, you know, one side or the other and pretty much what you expected it to be at the end?     

DR. FISHKIN:  I'm always surprised. And the moderators are trained never to -- never to give any indication of what they think and they are evaluated accordingly.  And they're there to facilitate a discussion.  And the advisory group, if it does its job, works very hard to get the strongest arguments for each of the options.  So that it could really go either way.     

And in that -- well, I could start giving you examples, but there's no time.  But the -- it's never precooked.  I mean, I wouldn't be involved.  Indeed, I would criticize any project that was precooked in some way.  There have to be -- and so that's why you want a variety of policy options which involve different tradeoffs and -- and distributions of benefits and burdens and costs and benefits and issues of equity and see how the public will wrestle with it in a -- in a serious way.     

DR. MICHAEL:  So I was going to ask you both about how you would operationally deal if you set up a possible experiment with the possibility of either bias or, worse, social harms when you ask certain questions in certain populations.  So I'm thinking specifically in places in Sub-Saharan Africa, if you were, as an example, to raise a question about access to care for vulnerable populations for HIV infection, such as men who have sex with men.  I know you and I both have spent some time, clearly, in Tanzania, so I think you know what I'm talking about there, or maybe a more stark example.  And I'm using polarized examples just as a -- you know, as an exemplar, but talking about land use and autonomy in the West Bank.     

DR. FISHKIN:  Well, my center is part of what's called the ResilientAfrica Network. We have a large USAID grant.  And one of the partners in that is in South Africa and we're in active discussions about an HIV-related project in South Africa.  But the -- we haven't done that one yet and -- but it's supposed to happen at some point.     

But we did do family planning in rural Uganda.  And this is controversial.  This is in a part of Uganda, the Mount Elgon region, where there were several very difficult aspects to it. The women are not getting -- many of them are not going to school at all.  And the -- they're married off at 13.  And there are multiple -- the population -- the Mount Elgon region, the average population density is 950 persons per square kilometer.  In Uganda as a whole, it's 175 persons per square kilometer.  So you can imagine how dense the population is.  There are households that we've found that have 55 people in the same household.  I mean, many, many women, many, many children.     

We did an almost perfect random sample of the population.  And there was strong increase in support for -- for enforcing the rule that -- the law that people can only get married at 18.     

And the question of schooling for girls, the advisory group came up with the idea of -- because the girls had to travel so far, that people were thinking they would get raped if they had to travel that far.  So the advisory group came up with the idea of village schools, schools right in the villages, one-room schools so the girls could at least get elementary education.  And that was one of the highest -- one of the most supported results.     

And for family planning, both the family planning and the education of girls was greatly affected by the fact that the district officials were consolidating the schools to make better schools, bigger schools, but where people would have to travel even farther.  And they were getting rid of the local health clinics so that people could have -- they'd have -- consolidate the health clinics so they'd have bigger and better. Perfectly reasonable from the standpoint of the district leaders.  But from the standpoint of the women in the villages, both were disasters because one of the main reasons the women would not engage in family planning was the -- they -- there were a lot of myths about it, a lot of anxieties.  And if they didn't have access to healthcare, they weren't going to do it.  And if they had to go two or three hours to get to healthcare, it was impossible.     

So one of the results when we met with the district officials and the, indeed, national officials from the Prime Minister's office about the results in Uganda -- I was just there a few weeks ago -- was that they were beginning to -- they were very open to reversing the decision. Instead of consolidating the schools, they were open to decentralizing the schools.  Instead of consolidating the healthcare clinics, they were open to decentralizing because with a deliberative poll, they were able to hear from the people and they had no idea that it would have -- the two decisions would have this effect on the girls and the women in these -- in these areas.     

So that's an example of a difficult set of issues in Sub-Saharan Africa where we can -- you know, I can actually report on the results and now we're -- the team from Makerere University in Uganda and my center are writing policy briefs which will be presented further, but we think we'll probably have those results.     

DR. WAGNER:  Scott, did you want to comment on this issue?     

DR. GUTMANN:  Could I -- could I follow up?  Because this example raises a general issue that would be applicable to any -- use of any deliberative polling.  And it goes back, actually, to something earlier you raised, Jim.     

So deliberation is actually seen as a kind of antidote to populism.  Right?  It requires people to stop and reflect.  It actually has a set of requirements built in that are very compatible with self-education as well as collective education.     

But -- so that said, I think there's no real controversy about that.  But there is both controversy and a question about how binding deliberation -- deliberative polling should be. And I just want you to reflect on two different models here.  One let's call the intermediary model, which is deliberative polling as a way of finding out what the people believe, but it is not the ultimate binding decision.  Right?     

And the second would be you take a deliberative poll and you bind yourself by the society, whatever -- at whatever level, whether it's local, national or international, binds itself to the results of this.  And I wonder if you would reflect on the pros and cons of each.     

I would say in -- I'll put my cards on the table that I -- for the purposes of bioethics decision making, I would like at least to have deliberative polling as an intermediary measure because if nothing else, you get a more educated sense of what -- we get a more educated sense, but whoever the decision makers are, they're not us, they're not the Commission, but it would give the policymakers a more educated sense of what the public believes and it also serves to educate the public in the best possible way.     

That said, if I think about the dysfunctionality of our Congress, for example, I would be hard-pressed to think that as an ultimate decision maker, the Congress right now is operating in a more functional way than a very well-designed deliberative poll.  But that would take a Constitutional convention to change.     

So I'd like your just reflections on are there any downsides to deliberative polling as an intermediary?  I don't see downsides to that. And what are the pros and cons of thinking of deliberative polling as more binding than that?     

DR. WAGNER:  Scott.     

DR. KIM:  I'll just say this is an issue that we've thought a lot about.  The fact is that if you take, let's say, the two models, just for convenience, consultative model, which is what you referred to as an intermediary, and then the binding -- policy binding model, practically speaking, if you're going to do the policy binding model, the money that comes to actually run the sessions will actually come from some source where people have certain interests.     

And even if it's nominally neutral, like the Government, there will always be certain forces, and there's great danger of just playing out the existing political power issues, just coming into and in some sense contaminating the process itself.  I really worry about that because there have been some complaints about that in bioethics in certain countries that tend to use much more public deliberation bases for trying to incorporate their policy.  So I think that's a real danger.     

So it's a -- what's the word?  A Catch 22 in some sense because to get close to the insurance that it's going to get incorporated into policy also comes at a potential cost.  And I think that's a real issue that one has to deal with.  And you could try to do that with rules at the very beginning, before you even start designing the studies, but whether the powers that be that would fund the studies would be comfortable with that would be a really very important issue to work out, so. . .     

DR. FISHKIN:  In "Why Deliberative Democracy," you have an account -- you have a couple of sentences about the binding alternative that's not as binding as what you just said.  It's more of -- I can't remember the exact wording, but it's more of an input intended to influence policy and maybe where the -- those who are receiving the input -- I'm mangling your prose, but those who are receiving the input will take it seriously or some words to that effect and that may be about the best you can do under most conditions.

I think that even cases which are pure consultation, two of the projects we did nationally in Japan with our partners at KOA University illustrate this.  The first was on reform of the pension system and the funders were actually the equivalent of the National Science Foundation or some group like that KAO got and we worked on the project with them.     

And it was just a scientific project, but it was also nationally revealed to the media and it revealed the logic of pension reform that the public would buy into, which turned out not to be the privatization that the Government was considering at the time, but, rather, an increase in the consumption taxes which wouldn't violate the privacy constraints people have because it's like a sales tax.  And the logic of that was so clear, it's what the Government ended up doing, but it was made part of the public dialogue with no preconditions.  It was actually a research project.     

Then a couple of years later, we did -- after the Fukushima disaster, the Government actually commissioned our friends at KAO, and through them, us, to do a deliberate poll on what to do about nuclear power after the disaster.  And it was a very interesting case where, first, they had town meetings, which we were not part of, where it's self-selected and where the activists yelled at each other.  Then they had online consultation where probably the same activists filled out the -- all of the comments, so they didn't know what to make of that.  Then they had polls and they didn't know what the polls meant or whether the people really were considering the issue.     

And then they did a national deliberative poll and it wasn't binding, but it was commissioned by the Government to consult in a highly visible way.  We had Asahi Shimbun.  We had NHK.  We had all the -- but it was -- and the Government at the time then struggled with this issue of whether to eliminate nuclear power or not because there were three different scenarios approved?  And the DPJ Government actually -- the Cabinet -- accepted that result.  But then they lost the election and the Abe Government -- they lost the election on other grounds.  The Abe Government is trying to reverse it and we'll see what happens.     

But the point is even commissioning, and they didn't say it was binding, but if the Government is supporting something and it's highly visible, the reasoning that is revealed by the public creates a great deal of pressure and a presumption that they ought to do it or take it very seriously or have a really good reason for not doing what the people want.  It's not just the result, but it's the reasoning that is revealed and the considerations for it.  So there's a kind of a grey area.     

DR. WAGNER:  Thank you.  That's --     

We're just going into overtime, but I want to sneak your question in, Christine.     

DR. GRADY:  I was just curious about the sort of enduring value of the process for the participants.  In other words, is there evidence to show -- you have data at one month, but further down the line in terms of attitudes or knowledge or desire to do it again kind of thing?     

And if you have a minute, I'd love to hear about the classroom toolkit, as well.     

DR. KIM:  Okay.  In terms of enduring value, there's two senses; one is enduring value in terms of how much the changed opinions are sustained.  And I think that's a bit of a mixed bag because if the topic is not something that's part of everyday conversation, I don't think it's realistic to expect, a year later, somehow that you'll have this -- because there's a lot of technical stuff that it's sort of maintaining while they're deliberating.     

But in terms of enduring value of people's valuing of the process, I think there are lots of studies that show that that's a very -- this is a very important engagement activation mechanism for people.  And there's a whole -- I would say a branch of almost like advocacy for deliberation and so forth that has that as the goal, so. . .     

DR. FISHKIN:  A couple of times we've gone back a year later and we find that there's considerable staying power to the opinions and the knowledge.  In fact, in one British case, they were even more knowledgeable a year later than they were at the end of the weekend.  So we think we sort of activated them to be interested.  And the -- as far as the -- the -- but they are back in their normal environment.  But we think we've sort of energized them to be more civically engaged.     

The classroom toolkit, well, we did a -- there's a -- there's a private school in Palo Alto or in the area called the Menlo School where we did a controlled experiment in the 9th grade where one-third had a deliberative democracy experience, one-third had no civic education, one-third had the civics education that they normally do, which we hadn't realized was by the prize-winning head of the school, the best teacher in the school.  And the deliberative democracy, which was participatory, did quite well in comparison to this prize-winning teacher.     

And so we're looking to replicate that controlled experiment.  And we've -- we haven't figured out -- we'd like to -- getting into the public schools is hard, but there's a -- there's a -- there are some -- we've done deliberative democracy experiments in the school.     

Saul Perimutter, who's a Nobel Prize winner at Berkeley, who's a friend of mine, has several times now developed a deliberative democracy -- a deliberative polling component for a couple of weeks in his big science and society class at Berkeley.  And the students love it.     

And last year, we did fracking as an issue.  And they developed the materials on either side for fracking.  And the students just engage in the participation, and they were very interested in the results.  And then I go down there and give them a lecture on other deliberative polls in other parts of the world and stuff.  So I think it has great potential in the school room and it would -- I'd really like to follow up on that aspect of our work.     

DR. GUTMANN:  Just one thing just occurred to me that's relevant to a recommendation we've made that Nita has really championed and we all concur with her.  So we've deliberated and went towards not a majority, but somebody who really compelled us with the reasoning, which is about the communication of scientific knowledge.  And I wonder if whatever else happens in deliberative polling, the actual engaging of people in giving arguments to one another helps -- is there any way of testing whether that helps teach the communication of what is often otherwise thought of as esoteric facts, but turn out to be very relevant to policymaking?  And is there some important -- important side effect of the process of deliberation in educating people to be able to communicate better?     

DR. FISHKIN:  Well, look, if the -- if the Advisory Committee does its work so that it's really clarified the arguments for and against different -- the contested arguments for and against different policy options and the assumptions behind them and communicated that in a way that people can understand, very often we'll even have a video version even in the United States much -- in addition to a written briefing document. But if they've done their work to prepare, then people will learn those facts; but most important, they'll learn the habits of an evidence-based dialogue.  Instead of just, you know, assertion and character assassination, instead of just mimicking talk radio and cable news, they'll get to the point of really saying, well, these are -- you know, here is the evidence.  These are the reasons.  These are the worries.  This is the risk, et cetera, et cetera.  And if it's participatory, they'll learn that whether they're students or adults.     

DR. KIM:  I think if there's anything I've learned from doing these sessions is that how unrepresentative public dialogue is related to the true capacity of people to engage in the kind of dialogue that you -- dialogue engage here.  And I know that because we've done really painstaking sentence-by-sentence analyses of transcripts of people deliberating.  And what you find is, you know, we've looked at things like back and forth.  We called it back and forth because we wanted our assistants to code for them.  And what you see is arguments going back and forth.  People have -- it's like they're in a philosophy class.     

And hese are ordinary citizens we've randomly selected, and they are fully capable of engaging in these issues when you create conditions for them to do so.  And there isn't -- there's established -- so, you know, one of the things we do is to ensure that we have some icebreaker and conversations so that people gain some trust with each other in the beginning -- at the beginning of the day.     

Once that becomes a pattern and they realize it's not a competition or trying to win an argument, you know, I have no doubt that the reason why we worry about this is that we get such a skewed view of how people argue about these issues in the media that's so out of line with how people could do it and fully retain the capacity to do it if you just create not so difficult situations to do this.

DR. WAGNER:  We've got to bring this to a -- bring this to a close and we are going to take a break until about 11:05, but not before thanking you so much for stimulating some great thoughts here. 

This is a work of the U.S. Government and is not subject to copyright protection in the United States. Foreign copyrights may apply.