VA Health Services Research & Development COIN Reviewer Orientation
December 12, 2012
Molly:And we are at the top of the house so at this time I would like to introduce our two speakers today. We have Dr. Sara Knight who is the deputy director of VA HSR&D and we have joining her Dr. David Atkins who is the director of QUERI and also the acting director of HSR&D. So at this time Sarah, I would like to share your screen with everyone. You will see a pop-up that says show my screen. Click that and we will be ready to go.
Sarah Knight:Molly, can you see my screen?
Molly:We can, go ahead and just open up the PowerPoint file down there at the bottom. Great, we are set to go.
Sarah Knight:Terrific. Welcome everyone and we are really delighted to have you joining us today and as Molly said we have Dr. David Atkins here, who is acting director of Health Services Research and Development,Carmen Corsetti and EricEnone, and Courtney Lindrup will be on the telephone so she will not be speaking but I also like to introduce and thank Carmen and Eric and Courtney as members of COIN team who have been corresponding with you and organizing many things, including assisting organizing our slides today. So I thank them very much.
Welcome and I want to let you know how much we appreciate your involvement in the review of applications for the new COIN initiative. One of the transformational initiatives of Health Services Research and Development, your time, your energy, and your insights are invaluable to us and we want to let you know that we highly respect your time and really appreciate your involvement. The COIN mechanism is new. And I hope that the review process and your involvement in this review will be exciting to you. We will be asking you to think about your search centers in some new ways. For today we would like to use this type of seminar to orient you to COIN. We will talk about what is new about the COIN. We will give you an overview of VHA and VA research. For some of you who are VA reviewers and VA investigators this will be very familiar to you. But for those of you who are working outside the VA we hope that this gets you oriented to some facts about the VA that may be important in evaluating the applications.
We are also going to talk a bit about why change a successful center system. As many of you know, we have had a successful Centers of Excellence system as part of our capacity building program in HSR&D. And our centers have produces many very successful investigators, helped produce impactful research, have mentored next generations of investigators over many years. So it has been quite a successful system. But we will talk today about why we are going to be building from that, not leaving it behind but really building from it to introduce some new ideas. We will give you an overview of the COIN initiative so you know the elements that you will be considering in the reviews. And we will also give you a brief, very brief overview of the CREATEs initiative. The reason why we are doing that is that there are elements of this initiative that will be embedded in the COIN. And so you need to have a little background about that. We will talk about the reviewer responsibilities, the critique template, just some highlights at this point, and key questions that we would like you to think about in the review and we will go over the scoring guide and those kinds of things.
We will have a second cyber seminar in January and that cyber seminar will get into more depth about the reviews. So I would like to begin and talk a little bit about what is new about the COIN. And so this table presents kind of a contrast of our older research centers and talks a little bit about how many research centers and health services may be organized. And then we will talk about how we are organizing the work on the COIN. So in our existing research centers researchers often collaborated. But they very frequently in our research centers were a bit siloed. And so they had their own research program that they developed as PIs using a mechanism called Investigator Initiated Research. And they often did collaborate with investigators at other centers. However, in COIN we are encouraging much more collaboration within a research center but also across research centers. In fact, that collaboration, that collaborative element, will be one of the things we would like you to think about in reviewing the applications. And in addition that will be part of the metric, how we will evaluate success of the COIN.
Now the way we looked at success and dissemination was that we would look annually at the publications and conference presentations that each center was producing. And that was a major element that we used to disseminate our research results. In COIN we are going to be looking at, of course the publications and conference presentations, but we, in addition, will be looking at engaged operational and clinical leaders, in VA and VA managers who have a vested interest in that COIN program. We believe that through these relationships with VA clinical leaders and VA operational leaders we will have a stronger pathway to translation to operational and clinical settings. And so of course we will want to consider the old mechanisms of dissemination but we will be adding some new ones.
Now our old centers were organized with themes and usually there were themes and groups of projects or groups of investigators whose work related to those themes – sometimes there was a very loose association between investigators and themes. And the projects within the theme usually addressed questions that were related to the theme. In the COIN we are looking for a much more strategic focus. So projects are going to be coordinated and highly focused and we are very interested in accelerating improvements and healthcare quality and outcomes. And we are looking for a much tighter relationship between themes and projects and the projects leading to improvement or projects that are developing methods. We are going to be looking for a much more strategic focus for our projects.
And how will we evaluate success? In the past we looked at the amount of research funding. We looked at the numbers of projects, the numbers of investigators even, and we looked at publications, the traditional metrics of success. In COIN we are going to expand that. Of course the traditional metrics are important. They are important to our investigators and their academic advancement. We do not want to change that. However, we are going to be looking more closely at the impact of the research on VHA practice, quality and outcomes, and not just for VA. We are going to be looking for a center’s ability to produce generalizable results and will impact health services research and development within VA but also beyond VA we will be looking at impact to the nation. So we are going to be looking at impact in an expanded way. We are currently organizing a committee to help redefine impact to some extent. And this is very much consistent with international efforts to expand the view from traditional impact factors to go beyond. And we will get the meaning of our research and what it means to improving healthcare and health outcomes. David, would you like to add anything to what I have said so far at this point?
David Atkins:No.
Sarah Knight:Terrific, thank you David. So just a few things about the VA healthcare system. Even those of you who are not in VA are very aware of the VA, I think. But just to reiterate some basic facts: the VA is the largest integrated healthcare system nationally. It serves almost eight and a half million enrollees, not all Veterans, but many Veterans are enrolled in the VA. And there are over a hundred and fifty hospitals or medical center facilities in the VA and many more outpatient clinics by current towns, over eight hundred outpatient clinics that are affiliated with these hundred fifty-two hospitals. You can get more information on the VA healthcare system by going to the link that is in this slide. And as many of you know the VA has had previous transformational initiatives such as one that occurred during the 1990s that organized the VA nationally into regional networks called VISNs. Currently there are twenty-one regional networks or VISNs.
Something that is really important for you to know in evaluating these applications is that the VA’s research program is intramural so that means that all investigators that have VA funding must have at least a 5/8ths VA appointment. They have to be more than a half time employee in the VA. The VA’s Office of Research and Development has a budget of over five hundred and eighty-three million dollars. It funds four research services ranging from basic bio-medical laboratory sciences to clinical research and cooperatives, studies and trials to rehabilitation research and our service which is health services. The budget for health services research and development typically has been running at about ninety million plus with additional funds for the QUERI program. And David, you might want to say a word about the QUERI program funding because it is a little bit different from the health services funding and also the mission is a little bit different.
David Atkins:The QUERI program gets money directly from the healthcare system rather than a research appropriation. And our QUERI centers are all located in conjunction with current centers of research. So you may see evidence of that synergy in the proposals that you are looking at. But because it is really a separate program, the aim of the proposal really should be talking about how the work funded by HSR&D and the research appropriation will have an impact. Their work and experience with QUERI may give them a slight edge in understanding how to work with partners. But it should not be factored in directly into the review of the COIN.
Sarah Knight:Thank you, David. So rather than presenting the whole organizational chart for the VHA, I wanted to just give you a sense of how we are located in the VA. So the Department of Veterans Affairs has three different branches including Veterans Health Administration. There is a branch that covers benefits for veterans, the VA, and a branch that covers funeral and cemetery services. And but we are in VHA. And we are in an area of VHA that addresses healthcare policy and healthcare services. So a major area of VHA that addresses clinical care and policy that is relevant to clinical operations. We are also located in a larger Office of Research and Development that I previously mentioned. And in health services research we have nine research portfolios currently. So we are a large and very vibrant service within ORD.
Just to give you a little bit of background about our capacity building program, I mentioned before we have had a very strong centers program that consisted of two components: one research enhancement award program or the REAP program. We had thirteen REAPS. And these were smaller in size compared to our Centers of Excellence. There were maybe three to five independent investigators involved in the REAP. And they were funded really to grow capacity in a medical center where small groups of investigators were clustered. And so you will see some of our COINs coming in will have been developed from REAP. In addition we have had a very long standing Centers of Excellence program, fourteen COEs funded recently. Some of them were very large in size. Some of them go back over twenty years. Often there were a minimum six to seven independent investigators. Very often there were many more.
And this program was originally funded to build and maintain health services research capacity and medical centers where many investigators were located, to train and develop the careers of the next generation of health services researchers. And to play a major role in national healthcare policy and research conversation. But why would we want to change this? It has really been a very strong program. We were happy with it. But we found that there were gaps in translation as many people have noticed outside the VA as well as in the VA. So we would produce very strong health services research and findings that were very important. But sometimes these findings were not implemented by the VA. And also investigators vision of issues and priorities important to the healthcare system sometimes were more narrow than might be best if we wanted to get a strong flow of translation. In addition VHA leadership in the field and in central office often weren’t that involved in research outcomes. They really did not have knowledge about what the researchers were doing and they did not necessarily have information about the outcomes in ways that they would really use the outcomes, research products. And then the design of projects often did not foster the translation. So they were not easily moved to clinical and operational settings.
And so for these reasons the – our leadership in HSR&D, actually before I joined HSR&D, David Atkins and Seth Eisen developed this initiative to really build on the past successes of HSR&D research. So we still want strong health services research that builds generalizable knowledge. But we also wanted to provide more effective mechanisms and incentives to reward research and the kinds of partnerships that we saw could be very helpful and especially with our experience from the QUERI program we realized that these partners were critical to translation. We wanted to provide ways of having these partnerships to ensure that research had the greatest possible impact on VA quality healthcare practices and outcomes for veterans and for the field of health services research even that beyond the VA.
So just to give you an overview now of the COIN initiative, so there was an eligibility criteria. So the COIN had to originate in a VA medical center. The center had to have at least five health services research and development or QUERI funded principal investigators. And as I said before, they had to be at least 5/8ths VA because they are an intramural program. And then they will need to go through our review process that will consist of two stages of peer review and that is to assess the potential of each center to have significant contributions to the VHA healthcare system veterans’ health services research and development in the nation. And then for those COIN applications where we determine through this peer review that you will be participating in, if we determine that there are additional questions that need to be answered before we make final funding decisions we will do what is called a reverse site visit is that we will invite the leaders of that COIN application to come here to Washington and to have some of their partners to visit as well to address these questions before we in central office make the final funding decisions based on your recommendation from peer review and these reverse site visits.
We want to emphasize something next about the budget. The previous system of Centers of Excellence and REAP, there were two levels of funding. REAPs received one level. There was limited amount of funding and the Centers of Excellence received a larger amount of funding. But the amount of funding that is simply based on our center being a COE or REAP and was not really based on research activities. And so we also recognize that there were some small REAPs that were actually powerhouses and really produced research of high value and addressed areas of need for VHA. And so we also wanted to have COINs that were small and large. So we have three levels of – three basic levels of COIN. But the COIN budgets themselves will be based on research activity. And the budgets will be variable. We intend to fund a minimum of fifteen COINs. We may fund more. And through this initiative we have already seen consolidation of some of our previously existing REAPs and COEs. And our review will help further this development, consolidation, and transformation.
So I want to say a few things again about we did not want one size. We did not want a one size fits all system. And so we expect that both small and large COINs will be valuable. And they may be valuable in different ways. We want you to think about in reviewing these applications. There will be some that are small, some that are mid-size, and some that are large. And so you will be thinking about the value of each COIN somewhat independently. And so you will need to adjust your expectations for the size of the COIN’s investigator team, the complexity and scope of the proposed strategic plan, the commitment of the partners, and the hosting medical center to provide resources as well as the academic affiliates. You will want to think about the relative importance of the work, size the focused areas of research. We will ask you to think about – this will vary with the size and the complexity of the COIN. And expect to see several multi-institutional COINs. Some of our COINs have already consolidated and I will talk about that next.