Working Towards Foundation Scheme Interpretation Guidelines
Canadian Institutes of Health Research
1
Working Towards Foundation Scheme Interpretation Guidelines
Last Updated: June 24, 2013
Preface
Over the past year, CIHR has engaged its research community on various aspects of the proposed design for the new Open Suite of Programs and peer review process.
We have heard from the community about the strengths of the proposed adjudication criteria, and where CIHR could further improve. In response, the criteria have evolved to better reflect the feedback received.
Discussions have been held with targeted members of CIHR’s research community to refine the proposed draft adjudication criteria and interpretation guidelines. Members of the various Reforms Advisory Working Groups represent diverse perspectives from all pillars of health research, and were selected based on their leadership in the field, and experience with peer review at CIHR and/or other funding agencies.
More recently, our discussions have expanded to include perspectives from other communities interested in conducting health-related research, such as the humanities, social sciences, natural sciences and engineering. Discussions with these communities are ongoing, and continue to refine the proposed Foundation Scheme interpretation guidelines.
We are now looking for your input to help us refine the most recent draft of the proposed interpretation guidelines for the Foundation Scheme. As you read through these guidelines, we ask that you please consider the following:
Stage 1: Caliber of the Applicant
- Do the criteria capture the essential qualities of a research leader, at any career stage? (Section 1.1.1, p. 6)
- Are there other examples of leadership, significant contributions or productivity that CIHR should include in its sample indicators? (Section 1.1.1, p.6; Section 1.1.2, p. 8; Section 1.1.3, p. 11)
Stage 2: Quality of the Program
- Is it clear that a review of the scientific and methodological details of the program of research is not required? (Section 2.1.1, p. 13; Section 2.1.2, p. 14)
- Are there additional assessment questions for the “Mentorship” section that should be considered? (Section 2.2.2, p.17)
General Questions
- Is there any language that would create barriers for applicants from any specific research field?
- How would you further develop the “considerations” sections?
An Introduction to the Interpretation Guidelines
CIHR is committed to ensuring that all applications to the Foundation Scheme grant competition receive a fair and high quality review. The Foundation Scheme interpretation guidelines are designed to support structured review, and are intended tohelp reviewers apply their knowledge appropriately and consistently.
This guide is structured according to Stages 1 and 2 of the competition and review process for the Foundation Scheme (Figure 1).
Figure 1. Proposed adjudication criteria for the Foundation Scheme at each stage of the multi-stage competition and review process.
Interpretation guidelines have been developed for each proposed sub-criterion, and are numbered and labeled according to the stage, criterion, and sub-criterion (e.g., Section 1.1.1 – Leadership). In the case where a criterion is not subdivided into sub-criteria, guidelines are provided for the established criterion (e.g., Section 1.2.1 – Vision).
Reviewers will be provided with the descriptors for each sub-criterion, as well as a detailed description of what the sub-criterion is intended to assess. Any additional information can be found under the “Mandatory Requirements” and “Considerations” sections for each guideline. Examples of sample indicators are also provided.
Overview of the Program Architecture: Foundation Scheme
The Foundation Scheme is designed to contribute to a sustainable foundation of health research leaders, by providing long-term support for pursuit of innovative, high-impact programs of research. Programs of research are expected to include integrated, thematically-linked research, knowledge translation and mentoring/training components.
The Foundation Scheme includes a separate stream for new/early career investigators with excellent training and early-career success, and will provide them with an opportunity to build promising programs of research.
The Foundation Scheme is expected to:
- Support a broad base of researcher leaders across career stages, areas, and disciplines relevant to health;
- Develop and maintain Canadian capacity in research and other health-related fields;
- Provide research leaders with the flexibility to pursue new, innovative lines of inquiry;
- Contribute to the creation and use of health-related knowledge through a wide range of research and/or knowledge translation activities, including any relevant collaboration.
Foundation grant values will be commensurate with the requirements of the research proposed, and will vary depending on the research field, research approach, and scope of program activities. Grant values will be within the range of approximately $50,000 to $1.5 million per year. Established investigators will be awarded 7-year grants, while new/early career investigators will be awarded 5-year grants.
The adjudication criteria described herein have been designed to select applicants that are best positioned to meet the aforementioned objectives of the Foundation Scheme.
The Foundation Scheme is supported by a three-stage competition and review process that will focus reviewer attention on specific structured review criteria at Stages 1 and 2 (Figure 2).
Figure 2. Proposed adjudication criteria for the Foundation Scheme at each stage of the multi-stage competition and review process.
Stage 1 focuses on the caliber of the applicant(s), in the context of the qualities deemed essential to meet the objectives of the Foundation scheme. These qualities are: leadership, ability to deliver significant contributions, and productivity. This stage also assesses the ability to define and articulate a clear and compelling vision.
Applicants deemed to be competitive at Stage 1 will be eligible to submit an application to Stage 2. Stage 2 focuses on the quality of proposed program of research,and will assess the program’sresearch concept, research approach, capacity, and supporting environment for the proposed program. Stage 3 involves the integration of the results of the Stage 1 and 2 reviews, and focuses on assessing applications that fall into the “grey zone” (i.e., close to the funding cut-off and demonstrate a high degree of variance in individual reviewer rankings). These applications will be assessed by a face-to-face interdisciplinary committee.
At Stage 3, new/early career investigatorswill be individually assessed and ranked against other new/early career investigators before having their final consolidated ranks integrated into the larger competition pool. CIHR is committed to monitoring the outcome of new/early career investigator applications.
Stage 1 Adjudication Criteria Descriptors and Interpretation Guidelines
Stage 1 focuses on the caliber of the applicant(s) in the context of the qualities deemed essential to meet the objectives of the Foundation scheme. These qualities are: leadership, ability to deliver significant contributions, and productivity. In some cases, there may be multiple Program Leaders for a single Foundation grant. These applicants will need to convincingly demonstrate synergy, and a history of co-managing programs of research, for some of thecriteria described below.
This stage will also assess the ability to define and articulate a clear and compelling vision.
Stage 1 - Criterion 1: Caliber of the Applicant(s)
Sub-criteria to be assessed under this criterion include:
1.1.1 Leadership
1.1.2 Significance of Contributions
1.1.3 Productivity
1.1.1 LeadershipDescriptors / Interpretation Guidelines
Does the applicant(s) demonstrate significant and effective leadership with respect to:
a)Being recognizedbytheir community;and,demonstrating a history of influential roles in his/herresearch field, such as advancingthe direction of a field or building networks/communities?
b)Establishing, resourcing, and directing a program of research, including managing mentoring/training, knowledge translation approaches and collaborations? / This sub-criterion is intended to assess the extent to which the applicant inspires others in his/her field of research and community, as well as the applicant’s ability to effectively directa program of research.
a)An inspirational leader is one who is widely recognized by his/her community, and has a considerable amount of experience in roles that require influence and ability in advancing the direction of the field and/or in catalyzing or mobilizing networks and communities.
Inspirational leaders may: have recognized advances in research and knowledge translation; be offered awards and acknowledgements; hold leadership roles and/or appointments; be sought for key speaking invitations; lead or participate in international collaborations, etc.
b)The applicant(s) shoulddemonstrate the ability toeffectively direct, manageand engage others to achieve research goals. An effective program leader would have experience:
- Developing and directing a program of research; multiple thematically-linked research projects; and/or national or international networks;
- Developing and implementing knowledge translation approaches for the dissemination, application and/or uptake of research findings;
- Building and managing collaborations (where appropriate).
- Developing and implementing mentoring and training initiatives for students, postdoctoral fellows, emerging scholars and/or new/early career investigators, including non-academic health-related professionals;
- Resourcing their projects and initiatives (e.g., securing funding, attracting and retaining research personnel, acquiring appropriate infrastructure, etc.).
Mandatory Requirements (if applicable)
Not applicable.
Other Considerations (if applicable)
General Considerations
The quality of the applicant’s leadership experience should be assessed first and foremost, with the quantity of experience assessed in the context of the applicant’s career stage and field of research.
In cases where more than one Program Leader is named, the assessment of leadership should consider the Program Leaders’individual and joint demonstrated leadership.
New/Early Career Investigators
CIHR defines a new/early career investigator as an applicant who, at the time of application, has assumed his/her first independent academic position (e.g., faculty appointment) no more than 5 years (60 months) ago. Trajectories from PhD to postdoctoral fellow to first academic appointment vary by research field, and may influence an applicant’s quantity of experience.
Note that new/early career investigators may show early evidence of leadership in different ways than established researchers, and that some of the more generic indicators outlined below may appropriately apply at a smaller scale.
Knowledge Translation
CIHR defines knowledge translation as a dynamic and iterative process that includes synthesis, dissemination, exchange, and ethically-sound application of knowledge to improve the health of Canadians, provide more effective health services and products and strengthen the health care system.
Sample Indicators (if applicable)
The following indicators are meant to be illustrative, and are by no means a comprehensive list.
Innovator in existing or emergent areas of research (“trail blazer”)
Leader of significant research endeavours
Leader of national and international collaborations, societies, associations, etc.
Leader of community-based research and or knowledge translation initiatives
Advisor in interdisciplinary or inter-organizational collaborations
Leader in the development or application of innovative training programs or approaches
Number and type of awards, honours, distinctions (note that these vary by research field)
Number and type of leadership roles/appointments (e.g., expert witness, committee, advisory group, network, key note speaker, appeals counsel,etc.)
Editorial or peer review committee service
Number, type and value of previous grants awarded
Number and value of salary awards received
For new/early career investigators, specific indicators to capture early evidence of leadership competency would also include:
Evidence of an emerging profile of innovation or 'trail blazing' (e.g., development of new research methods)
Strong promise as a leader of significant research endeavours
Significant involvement in key committees, national and international collaborations, societies, associations, etc.
Participation in advisory groups in interdisciplinary or inter-organizational collaborations
Central role in the development or application or innovative training programs or approaches
Nomination for awards, honours or distinctions
1.1.2 Significance of Contributions
Descriptors / Interpretation Guidelines
a)Has the applicant(s) significantly advanced health-related knowledge, health research, health care,health systems and/or health outcomes?
b)Has the applicant(s) engaged, trained, and/or launched the careers paths of promising individuals in research and/or other health-related non-academic fields? / This sub-criterion is intended to assessthe extent to which the applicant’(s)’ contributions have a tangible benefit, or positive influence, related to health. These contributions could include advancementsin different health-related areas, as well as contributions to maintaining a sustainable foundation of health researchers.
a)Advancements should relate to the applicant’s research field. They should be evident, and mayfallwithin one or more of the following health-related categories:
- Significant contributionstothe advancement of knowledge, which would include advancesor improvements to the current thinking in the field of research that have been sustained.
- Contributions to health research, which would include broad-scale or transformative changes to current research practices and approaches.
- Contributions to health care, which would include improvements in diagnosis, treatment, and prevention of disease, conditions, disability, injury, and other physical and cognitive impairments.
- Contributions to health systems,whichwould includeimprovements in the organization of people, institutions, health information,health policies, and the human and financial resources to deliver health care services.
- Contributions to health outcomes, which would include improvements to the health and/or quality of life of individuals or populations.
- New economic opportunities stemmingfrom the application or commercialization of health research.
Mandatory Requirements (if applicable)
Not applicable.
Other Considerations (if applicable)
General Considerations
To address this criterion theapplication includes theapplicant’(s)’ top five (5) most significant contributions in the last five (5) years.
In cases where more than one Program Leader is named, the assessment of significant contributions should consider the Program Leaders’ individual, as well as any joint, contributions.
New/Early Career Investigators
Contributions from new/early career investigators to a sustainable foundation of health researchers may be different from thoseof established researchers. Accomplishments may focus on efforts to build capacity (e.g., ability to attract students/trainees/emerging scholars, developing and implementing innovative mentoring/training curricula). Note that some of the more generic indicators outlined below may not yet be measurable at this stage, or may appropriately apply at a smaller scale.
Sample Indicators (if applicable)
The following indicators illustrate different types of significant contributions, and are by no means a comprehensive list. The applicant(s) has been asked to describe his/her contributions according to the following headings. Note that the type of contributions listed may vary by field of research.
Contributions to the Advancement of Knowledge
Important advances in understanding (e.g., informing controversies, examining assumptions, improving theoretical understanding in health professional education and in integrative research)
Significant advances in knowledge (e.g., pathogenesis, cell differentiation, biological pathway, gene-environment interactions, barriers to medication adherence, synthesis)
Significant impact on a current area of research (e.g., paradigm shifts)
Contributions to Health Research
Development of new areas of research
Development of theoretical frameworks/models
Development of research methods/techniques
Benefit of sustained change in research practice (e.g., ethics considerations)
Impact of new research approaches (e.g., multi-disciplinary approaches, horizontal (system-wide) research focus, etc.)
Contributions to Health Care
Evidence-informed health care practices
Evidence-informed health promotion and prevention programs
Evidence-informed treatments, therapies, interventions or disease management approaches
Evidence-informed health technologies, tools, diagnostics and devices
Increased patient satisfaction
Reduced burden of illness
Contributions to the Health System
Evidence-informed policies (health or other) that positively impact health
Efficiencies from improved resource allocations or practices (e.g., cost savings, reduced wait times)
Improvements in health care data, information management and infrastructure
Reduced health inequalities in a specific population
Effectiveness of new or improved health care services, practices and regulations
Improved health professional education policies or education system
Cultural, organizational or pedagogical changes that have improved health professional competency
Contributions to HealthOutcomes
Improvements to the health status or quality of life of populations
Clinical outcomes of health research interventions
Incidence and prevalence of chronic and acute illnesses
Changes in culture, attitudes andbehaviour that improve health
Public health awareness and education (documentaries, classroom/course material, and film)
De-bunking stereotypes/myths
Contributions to Economic Opportunities
Business opportunities created
Creation of jobs from spin-off companies or from expansion of existing company
Sales/revenues generated or licensing returns
New products, processes or services commercialized
License or royalty agreements
Contributions to the Foundation of Health Researchers
Number and types of trainees mentored
Increased ability to attract international students/trainees/emerging scholars
Implementation or broader uptake of innovative mentoring/training approaches
Career success of mentees/trainees
Success of mentees/trainees in working within both academic and non-academic spheres
1.1.3Productivity
Descriptors / Interpretation Guidelines
a)Have the applicant(s) demonstrated an outstanding level of research outputs, based on prior work?
b)Has the applicant’(s) previous work generated high quality research outputs? / This sub-criterion is intended to assess both the quality and quantity of research outputs generated by the applicant(s) in the last five (5) years.
Mandatory Requirements (if applicable)
Not applicable.
Other Considerations (if applicable)
General Considerations
The quality of the applicant’(s) research outputs should be assessed first and foremost, with the quantity of research outputs assessed according to the applicant’(s)career stage, and field of research.
In cases where more than one Program Leader is named, the assessment should consider the Program Leaders’ individual andjoint productivity.
Sample Indicators (if applicable)
Research outputs vary by research area, and may include:
Collaborations or networks
Communication and knowledge translation products or resources
Competency frameworks for health professionals
Conferences and conference proceedings
Consultations
Databases and evidence repositories
Intellectual property claims
Invention disclosures
License agreements
Media interviews (e.g., television, radio, print)
Patents (filed or obtained)
Policy briefs
Presentations at public forums
Prevention or intervention programs
Products
Professional practices
Programs or services
Public information, resources, and tools
Publications, peer-reviewed or other (e.g., theses, journal articles, books, book chapters, workshop reports, synthesis reports, dissemination reports, conference proceedings)
Software
Spin-off companies
Standards and guidelines
Theories, models or frameworks
Tools, techniques, instruments, procedures or methods
Training approaches/curricula
Stage 1 - Criterion 2: Vision