TRANSCRIPT, Meeting 7, Session 4

Date

November 16, 2011

Location

Boston, Mass.

Download the Transcript

Transcript

James Wagner:  I’ll tell you what.  Maybe we will get rolling.  Dr. Hauser will join us in just a moment.  Welcome back everyone and Commissioners. It is good to be seated with you in this room, as well as in the [Inaudible] room.  It is good to remember that what we are trying to answer is the charge to determine if federal regulation and international standards adequately guard the health and well-being of participants in scientific studies supported by the federal government.  But to pursue this, I remind the Commission and maybe to the news of many of you out there I thought it was important to understand the existing landscape of existing studies supported by the federal government – the piece that we are supposed to comment on.  When the staff set about to do that, they requested data on research involving human subjects from eighteen departments and agencies.  Through that effort, they learned not only about the landscape of federally supported human subject research but also about the variability among agency systems for tracking human subject research they support.  In fact, I think we were all struck by how much the information is not available or at least not available all in one place.  Given the increasing, and appropriate, call for public access to information about human subject research, we found the work very relevant to the overall discussion of transparency and accountability.  So one of our subgroups that was assembled has been on transparency and accountability and Christine Grady will talk us through some of the deliberations of that work leading up to possible recommendations.

 

Christine Grady:  Thank you, Jim.  First, I want to recognize – and I think everyone recognizes – that there are substantial societal rewards but also substantial societal investments in research.  I was struck this morning, we had $49 billion in 2010 from pharma and NIH budget is $30 billion.  So, that’s a lot of money.  So the public trusts all of this research to generate knowledge that is useful and has a reason to expect some transparency and accountability in the pursuit, especially in the use of public resources.  I think there are examples that we have seen of risks associated with non-transparent conduct of research, or research that is not done in the limelight or sunlight.  There has also been a call over recent decades for increased public access to publicly funded science, especially biomedical science, and the international research panel that we worked with emphasized the need for greater transparency and making information about medical research, especially greater than minimal risk research, available online. 

            I think it is worth our recognizing that there have been many important advances in the last decade or so in this regard.  Clinicaltrials.gov includes basic information about clinical trials that are studying drugs, biologics and devices and is publicly accessible information that is included for privately and publicly funded research on drugs and devices under FDA jurisdiction.  Many other countries have public registries for clinical trials:  Brazil, India, Europe, others.  The WHO has an international clinical trials platform that is voluntary.  There are other sources of public information about federally-funded research.  There is USA Spending which has information about federally funded grants and contracts; there is a relatively new health research web that was developed by the Council on Health Research for Development, or COHRED, which has information not only about research studies but also about IRBs and research ethics review committees and other sources of support for research ethics in places around the world.  Recently, PubMed Central has become a publicly accessible database of publications that result from federally-funded investigations.  So there are a number of things that have happened and some federal agencies have developed systems that are available to the public.

            As Jim already mentioned, the Commission in an effort to evaluate the adequacy of protections wanted to get a picture of the landscape, the volume and the scope of research supported by the federal government.  To that end, asked each of the eighteen federal agencies that are covered under the common rule to provide project-specific information regarding their human subjects research portfolio in 2010 and some trends prior to that.  We also asked for what was available from the agencies including the title, the countries that the research was being conducted in, the number of participants, funding information, things like that.  I think it is worth noting that one hundred percent of the agencies that were asked for data provided data to us about their portfolios.  But as Jim already alluded to, the way they keep their information is very different from one agency to another.  Their systems are variable; some are more complete than others in terms of what they include.  There are certain kinds of fields that are particularly hard to get from most agencies.

            With an incredible effort on the part of the staff, these data are still being analyzed and put together in a usable way.  But a couple of things that we do know already is that the US government supports tens of thousands of human subject research projects every year.  Most of them are biomedical, but not all of them.  In fact, the range is quite interesting and is something that there is not a lot of available information about for the public.  The largest agency is HHS but all of the other common rule agencies do some human subjects research. 

            Putting all of this together and trying to make sense out of the need for more publicly available information about research that is being done by the federal government, the Commission is coming to the conclusion, I think, that the expanded public access to certain information about federally supported research is both desirable and can be accomplished.  But rather than suggesting the creation of a new central government-wide database, we are thinking about recommending that the existing systems at each agency be further developed and improved and include a minimal set of fields.  Then each of those could then be available through a central web-based portal that is also developed by the government.  So the public would have access to information at each agency through this central web-based portal. 

            That is one thing that we are thinking about recommending.  The other is in recognizing the need for information and data regarding how well the current standards and procedures do in fact protect human subjects and recognizing the current limited data on effectiveness of protections that exist the Commission is thinking of recommending federal government support for research and evaluation and other systematic approaches to evaluating the effectiveness of human subjects protections.  So that is where we are.

 

James Wagner:  And Lonnie and Nelson were part of that group.  Anything either of you would care to add?

 

Christine Grady:  Yes, Lonnie and Nelson.   And Michelle.

 

Lonnie Ali:  I think in the group discussions we came to the conclusion that, especially given the information that was given by the landscape data, that there was a lot of information that was not available, which to me was very surprising considering that a lot of this has to do with individuals and people and having respect for those individuals and people there was no kind of accountability with regards to some of the research being done.  And I think in some instances, and correct me if I’m wrong Nelson and Christine, there were some agencies that didn’t actually have data at all or a place where they could go and pull this data easily for us to be able to actually look at.  When you talk about transparency, and I don’t know how deep we as a commission we want to go as to exposure of information, but public access to me is very important—for people to be able to go online and be able to access this data.  That was a suggestion with regard to the central web portal.  We understand it would be very difficult for everybody to be able to try to do something that conformed to one model.  This would give us an ability to access their information.  The thing we struggled with is how much information we want, what kind of fields we want that should be reported.  That is one of the things that we are grappling with now.

 

James Wagner:  Nelson?

 

Nelson Michael:  Dr. Medford this morning used an analogy of railroad gauges that are different and between agencies the railroad gauges are completely different.  So our deliberations reflected on that reality and attempted to find a feasible middle ground where transparency could be achieved and achieved in relatively short order but without putting into place a framework that would be very difficult to implement.  So we wanted to the good not to win and not be a situation where we achieved something that was perfect and would kill the good.

 

James Wagner:  It is interesting, Dr. Medford also mentioned – and I might have heard a different number – but estimates of the relative size of the industrial research initiative funding levels – I’ve heard estimates sometimes twice what’s going on in NIH.  Do we feel that we can give assurances to the President, or how do we feel that we can answer this charge if we don’t also give assurances about having that sort of transparency in industrial research?  And what are our assurances there?

 

Christine Grady:  Privately sponsored research, that is biomedical anyway, and investigates drugs and devices and biologics is already required to be part of clinicaltrials.gov.  What is interesting about the data that we have been able to collect from federal agencies is that the federal government supports, and understandably looking at what the agencies are, human subject research that goes well beyond biomedical.  Lots of interesting other areas like education, for example.  So this recommendation would actually expand the amount of publically available information about research in an extraordinary way that is beyond biomedical.

 

James Wagner:  Nita.

 

Nita Farahany:  First, I think these recommendations are important and very good ones and I think it is right that we don’t want to enforce any particular model agency by agency because, as you put it Nelson, it would put the perfect the enemy of the good.  While I agree with that, that we don’t want to make it cumbersome, it does fit within the model of the federal government right now to try to streamline different web-based approaches across the government.  So, for example, USAgov.search is trying to have a uniform search engine across all the different agencies.  And there is an across the federal government standardization process for web-based processes being developed right now and being encouraged.  So I don’t think we should mandate it but I do think in our ideal world what would happen is that it actually would be a compatible system from agency to agency.  And to the extent that agencies don’t already have a system and are being encouraged to put into a system some sort of minimum criteria, it may make sense just as the government has outsourced contractors to develop something like USAgov.search, that they similarly do something to help agencies to develop a system by which they can enter this information.

 

James Wagner:  Raju?

 

Raju Kucherlapati:  I wanted to make two observations.  One, Christine, the recommendations look very good.  But I wanted to make this comment.  With regard to the goals to why people do research and what the ultimate outcomes are, whether academic research or public research, private research, one is that they want to publish papers or they want to be able to do clinical trials for drug approval or some other nature for commercial interest.  And certainly all of the medical journals now require that in order for your paper to be considered, you need to have to register the trial in clinicaltrials.gov.  Otherwise, they will not even consider the manuscript.  So that is an important driver for people to register.  As a result, most of the studies are indeed going into clinicaltrials.gov. 

            The second one is actually an interesting question for us at the Commission.  We have argued many times that having this information – I support the recommendation  -- the question, that having this information is going to help us in trying to assess whether or not all of these are being ethically conducted or not.  So one of the questions is, if we have the data now, do we have an answer now to that question as to whether or not, with whatever data we have available whether we can address that issue?

 

Amy Gutman:  Let me begin to answer that question because I think it is one that we have all asked ourselves.  Okay, what would give us the basis for assuring the President that every human subjects research, every trial – whether it is clinical or otherwise – above a minimal risk standard is being conducted ethically?  If it were the case that just having the database would be the magic bullet, then we would have one recommendation here and run with it.  And it is not.  So Raju’s premise is absolutely right.  Having this database will not in of itself guarantee that research is ethical.  However, it is certainly not sufficient, but is it necessary?  I would say absolutely yes.  The landscape project that our staff undertook has been going on for months now and it took months and the goodwill of all the agencies to get the following information:  study title, performance, country or countries, number of participants, and funding information.  The agencies as they are now set up, their databases do not allow them to have quick access.  So it is a necessary not sufficient and until you get the ability to have that degree of transparency you cannot get the accountability that would enable not only our commission but anybody who has a right to know to basically say, “clinicaltrials.gov” is everybody actually registering who needs to register there?  What other kinds of research is being done that poses more than minimal risks to people?  I think it is very important for us not only to make this recommendation but also to be clear that this is just a first step.  It is not the last step and it would never be a sufficient step.  But it is a step that directly answers many people’s legitimate concern about transparency.

 

James Wagner:  Dan?

 

Daniel Sulmasy:  I add to that that it doesn’t answer the questions but in part it tells us where to look.  For instance when we find out how much is done by the Department of Transportation or Education.  We haven’t even been thinking about as a country as potential places where human subjects research is being conducted under the aegis of the federal government.  Second it gives us an idea of the magnitude.  So if we say it is one percent, it is one percent of what?  We had no idea previously.  So I think those are important aspects of the empirical data.

 

Amy Gutmann:  When Christine says tens of thousands, what we now know and we are still counting, is it is over fifty thousand last year.  Over fifty thousand studies.  And that’s, in and of itself, -- and then we don’t know the number of human subjects, but when we know that we’ll actually have some sense of how many people.  So if nothing else, if you look at the reporting at adverse incidents you’ll have a sense of what the percentage is.  If it is 100th of a percent, that is much different than if it is one percent of the number that are human beings.  So this is a very important first step and I think we are all somewhat surprised, and I say somewhat to different degrees depending on how close we were to the operation of different agencies.  It is I think one of the aspects of the report that will come as a surprise to many people. 

 

James Wagner:  Christine there are elements that the fuller Commission can help the group with as you go forward.  Lonnie mentioned you are wondering, you’ll be making a recommendation what the minimal dataset might be.  You have heard Amy mentione the number of participants, also the adverse outcomes.

 

Christine Grady:  It would be great if members of the Commission have ideas or opinions about what the minimal dataset should look like.  At the moment, we are thinking about Title, PI, Funding Source, and Location – where the study is taking place.  We could add number of participants, but that is something that has been particularly difficult at least in the first round of collecting data, very few agencies have that information.  And dollar amount.

 

Amy Gutmann:  We could put it as here is what we think is a practical ideal, not one that is in our utopia.  But we think, given the acknowledged responsibility of government to the public trust and to protecting human subjects, we think it would be good to know the number of human subjects and the kind.

 

James Wagner:  Nita, then Lonnie.  Is that okay?

 

Nita Farahany:  One additional thing that I think would be really great to add to that list, something that has frustrated me as we have been collecting the data and unable to get is the nature of the study.  So even if it is a rough set of categories that says here are the types of studies that might be undertaken.  If we just have a title and we know how many participants there are, but we have no idea what the nature of the study is, it seems hard to provide true transparency, true oversight or even to know which studies you want to scrutinize in any more detail.  So I think it is something harder for us to do retrospectively because it requires us to data gathering on things that have already happened; prospectively, it seems like something that should be easier to capture. 

 

James Wagner:  Lonnie?

 

Lonnie Ali:  I was just going to say, I think it is important, too, as a group that we consider when this reporting takes place.  Does it take place at the beginning of the trial when it is first introduced, or is it someplace midway through.  Because one of the things we might want to consider is adverse impacts and what happens if there is something that needs to be reported that has occurred.  So I think it is important that we may need to put into that recommendation when this reporting takes place.

 

Amy Gutmann:  Very important, I think, so we’ll see that it be done at the approval of the study because we have adverse incident reporting which comes retrospectively once there is an incident.  This should be a prospective report to know what is in process.

 

James Wagner:  It is also more likely to be welcomed by the agencies if it is upfront and it is not done on a continuing basis, particularly if we have in place mechanisms to do the other.  Anita?

 

Anita Allen:  One of the reasons I liked this proposed recommendation is because we do have a national public policy of fair information practices and freedom of information access.  We also have from President Obama, an initiative announced in January of 2009 toward more transparency and accountability in government.  So I think the President will welcome this particular version of transparency and accountability.  I do have, however, two concerns.  One is a concern about who would have access to the data and whether our recommendations should proactively cite any costs or any downside to making this information available and even if it is as limited as a title of the research, location, funding source and participation level, there may be sensitive issues around health privacy that would need to be concerned about and I wonder whether we have any thoughts about those kinds of things?  Concerns about individual privacy or concerns also about who in the general public will have access because, as you know, it is one thing for a database to, in some sense, to be public.  But, if it is public, but obscure because it is not really available to the general public it is a different kind of a story.  Do we intend this to be a widely available public resource or do we intend it to be information which the government can use in its own internal policy making and strategizing?

 

Christine Grady:  I’d like to respond to this.  I think our idea was to have it widely, publicly available, but I think Lonnie’s question is very important.  Depending at what point in the trajectory the data is entered into this – presumably from the agency’s perspective, it is already in there at the level they make a funding decision.  So that is the time that is most likely to be available to be available at a broader way.  So every time we think of adding something like the nature of the study or other things, then the question of what are the costs and what are the pitfalls of doing it come up and we should not only recognize those as we discuss it but we should recognize that as we write it up as well.  I don’t know how we anticipate the cost because each agency has a different system as it stands.  Some are much more complete than others.  So the agencies that have very little are going to have a big cost in ramping up even minimal datasets.

 

Anita Allen:  I didn’t mean by costs simply monetary costs.  I meant, for example, if some political groups in our society decide they don’t like the fact that X dollars are going to X type of research, it might go to the politicization of human subject research in ways that the scientific community would be very unhappy about and some of us would be as well. 

 

Christine Grady:  So what would you recommend in that regard?  Don’t list them?

 

Anita Allen:  Not at all.  Again, I am in favor of the transparency and accountability.  I just think we should be very honest and open and proactive in identifying for the President the kinds of costs and benefits and issues of all sorts which will result from this greater degree of openness.

 

Christine Grady:  Okay. 

 

James Wagner:  [Inaudible] Did you have your hand up on this?

 

Raju Kucherlapati:  If I can just follow-up on this?  There is already an example, there is a database at the NIH called CRISP that contains all of the information about all of the grants that re made at the NIH that includes the title, it includes the abstract which is prepared for lay people, and it is also possible to obtain the duration, the amount of money and so on and so forth.  That has been in place for a long period of time.  So there is a lot of experience in dealing with the kinds of issues that Nita and Anita brought up.  I think we should examine if that is a model that we can use because we have experience with it and it is publicly available.

 

Amy Gutmann:  And it is a model.  The problem is at the current point, and this is given the monetary cost, time and priorities, to advise the government to require every agency to have that exact same model where the other agencies have tiny research budgets compared to the NIH would be a mistake on our part.  It would let the perfect be an enemy of the good.  Whereas to recommend that they have a good system of prompt reporting which protects privacy of individuals  and the names of subjects are not being identified.

 

Daniel Sulmasy:  One other response on this topic, for the kinds of data maybe that Nita was asking about, once a database is available, it doesn’t preclude the possibility of doing sampling studies within those and multiplying that.  It is another advantage of having this database to begin with because then you do that kind of sampling.  Then a question for Christine:  I know you have spoken to us about this before, but for our mutual benefit and that of our public, to talk again to talk about the distinction about adverse event reporting and this set of recommendations for a database and why the subcommittee thinks it ought to be separate?

 

Christine Grady:  Earlier we had a presentation that Barbara gave us on recommendations in the ANPRM.  One of which is to develop a common database for adverse events and unanticipated problems in research so that there aren’t more than one reporting system for than when a researcher does his or her research.  We envisioned that as distinct from this for several reasons.  One is this is a way at the agency level to make transparent what they are funding, not only in terms of dollars but also types of research.  I like the idea of sampling within, but not results and not what happens later.  Whereas side effects and unanticipated problems are very important for understanding how we are protecting people and what the risks are of participating in research and having them centralized in another way is very important.  But it comes in a different time in the research process time than this presumably would and it is reporting that is done by a different category of people, so the investigators report side effects and unanticipated problems whereas in this case we are asking the agencies to report.  So they are really distinct kinds of reporting systems.

 

James Wagner:  Thank you.  I failed to ask the audience, we do have mechanisms by which you can submit questions or comments and I guess I’ve got one coming up right now that we can look at right now.  But please do. 

            Does that get to your—essentially what you are saying is that it is important, but there are other mechanisms in place and there will be data provided from another source, than what you are talking about up front.  Russell Medford, actually.  He has a suggestion.  Consider applying a unified and effective model going forward.  Funded programs must adhere to database.  I’m going to need you to stand up and explain that. 

 

Russell Medford:  Do I get the ten-minute timer?

 

James Wagner:  No, as a matter of fact.

 

Russell Medford:  I think you are wrestling with a real variable problem.  You have a legacy database and fifty thousand trials, incompatible data systems to actually get the data.  So my suggestion might be, and it might be more acceptable to the agencies, is build a more effective and unified and bought into database approach on a prospective basis, saying, “from now on, on a given date, the data must be input in a certain way .” And separate the legacy problem from the go forward problem, so you have a two step.  Everything from some date would go into a universally accepted common database and meet all the requirements and I think you need more than just the limited information that you can get and address the legacy issue separately. 

 

James Wagner:  I think it is probably intent. [Inaudible] Nita?

 

Nita Farahany:  I think, just as we have been we’ve been discussing it, we’ve been discussing it as if we want to allow agencies to preserve their existing databases.  I do think we want to clear when we develop our recommendations and develop a draft to say that something like a unified model if it were possible to develop, something like the unified search system and all of the processes the government is developing to try to unify systems across government, it may be a lot less expensive.  A single database that is developed and given to each of the agencies might be the most efficient prospective system going forward.

 

James Wagner:  Of course, our history on that is kind of discouraging but I think — .

 

Nita Farahany:  Recent history is quite encouraging.  There are amazing efforts that have happened in the past couple of years to try, across the federal government, to improve the web-based systems and to unify the systems, particularly for this purpose – for transparency.

 

James Wagner:  I’m encouraged.  I would like to take your theme and suggest as a friendly amendment that the recommendations be that going forward, to take your point, whatever databases the agencies are using would include these data and we would have them available, but then holding up that the obviously the ultimate would be this more unified piece.  So if we could take prospective before unified, I think we might get a more satisfying result.  Thanks.

 

Nita Farahany:  Thank you for that helpful comment.

 

Anita Allen:  But is part of our agenda to uncover any existing misconduct that we might miss by simply recommending a forward looking integrated data system?

 

Amy Gutmann:  Part of our agenda here is what Dan earlier said.  You would want to do a sampling of human subjects research that is out there.  And unless you have a base on which to do that sampling, you can’t really have a statistically significant sampling exercise.  So that is part of it.  I think getting some way that data is collected at the agency level so that it can be made part of a unified system, whether it is a two-step or a one-step process, the dance itself doesn’t make as much of a difference as the final outcome which is that there is a publicly accessible database. 

 

James Wagner:  And what we recommend get unified immediately is the minimum dataset and then the process can be —  I’m pleased Nita to hear your encouragement that we can expect to move on that.  Not having other questions coming forward and Lonnie, Nelson and Christine, are there other ways that the Commission can help you as you go forward? 

 

Lonnie Ali:  I think given Raju’s comment, maybe we do look at that for some guidance on what Anita said and, Anita, maybe given your background call on you as well with regard to that protection of information.  Because it is something to consider.

 

 

This is a work of the U.S. Government and is not subject to copyright protection in the United States. Foreign copyrights may apply.