An Evaluation of the Carbon Farming Futures Programme

An Evaluation of the carbon farming futures programme

June 2017

Table of contents

1Executive summary

2Introduction

3Carbon Farming Futures

3.1Background

3.2Carbon Farming Futures Design

4Evaluation methodology and scope

4.1Evaluation approach

4.2Key evaluation questions (KEQs) and data sources

4.3Evaluation activities

4.4Limitations

5Administration, implementation and delivery

5.1Programme design

5.2Grant selection and application process

5.3Project design

5.4Project focus

5.5Programme and project funding

5.6Performance management and monitoring

5.7Variations

5.8Collaboration

6CFF impacts and outcomes

6.1Achievement of short term objectives

6.2Links to the CFI and ERF

6.3Productivity impacts

6.4Programme legacy

7Conclusions, recommendations and lessons learnt

7.1Implementation, administration and delivery

7.2Programme impacts and achievement of objectives

7.3Lessons learnt

Attachments

Attachment A – Consultations

Attachment B – Key Evaluation Questions

Attachment B - Carbon Farming Futures Programme Logic

Department of Agriculture and Water Resources – Evaluation of Carbon Farming Futures / grosvenor management consulting / 1

1Executive summary

Carbon Farming Futures (CFF) was composed of three interrelated grant programmes administered by the Department of Agriculture and Water Resources between 2012and 2017. Through this programme, more than $145 million of funding was allocated across 200 projectstoensure advances in land management technologies and techniques for emissions reduction and adaptation that will lead to enhanced productivity and sustainable land use under a changing climate.

CFF was intended to create new opportunities for land managers to enhance productivity, gain economic benefits and help the environment by reducing greenhouse gas emissions or increasing carbon sequestration. Such actions can also increase the land sector's resilience to climate change, protect Australia's natural environment and improve long term farm productivity. CFF was also designed to assist in the development or improvement of tools and methodologies that farmers can utilise to participate in the Australian Government’s Emissions Reduction fund (ERF[1]).

Grosvenor Management Consulting was engaged by the department in 2016 to conduct an evaluation of CFF and its three constituent grant programmes; Filling the Research Gap (FtRG), Action on the Ground (AotG) and Extension and Outreach (E&O). This evaluationwas undertaken to understand the overall success of CFF andeach of the three grantprogrammes, addressing the following Key Evaluation Questions (KEQs):

to what extent have the objectives and outcomes of the CFF programme(s) been achieved?
how effective and efficient was the implementation, administration and service delivery of CFF programmes and projects?
what are the lessons learnt from the CFF programme(s)?

The evaluation was informed by the desktop review of all funded projects in addition to consultation with key stakeholders. Both programme and project level analysis was undertaken to understand:

the success of each funded project and its contribution to the achievement of programme level outcomes and objectives (project level evaluation)
whether the objectives of each programme (FtRG, AotG and E&O) have been achieved (or are being achieved) and whether each programme was well implemented (programme level evaluation)
the contribution of each programme (FtRG, AotG and E&O) to the objectives and overall success of CFF.

Implementation, administration and delivery

Overall, the design and deliveryof CFF was appropriate to achieve the intended outcomes and was well supported by the department and grantees. Very few concerns were raised about the administration of CFF during the evaluation, with the vast majority of projects delivered on-time and in-budget.

There was a high level of satisfaction with the administration approach and model applied by the department. Processes for the initial application and management of the projects typically aligned with grantees expectations for large Commonwealth grant programmes, particularly within FtRG and AotG. Despite this, some concerns were raised in relation to the level of detail required in the E&O progress reports and the extent to which the department was involved in the review and finalisation of extension materials. Notably, grantees found the requirement to submit all materials to the department for review and approval, as well as need to attach all materials to the progress reporting (even when publicly available), as a burden.

Clear linkages were identified between the programme design and intended outcomes, with support expressed for the overall design and model. The initial programme design envisaged a phased approach to programme delivery where feasible topics and practices could evolve from research under FtRG, to trial under AotG and ultimately to extension through E&O. The reduction of the CFF funding and overall programme period following Government policy changes inhibited the adoption of the phased approach, with an insufficient number of rounds to allow practices and technologies to develop and evolve across the three programmes.With all programmes commencing simultaneously, the initial AotG and E&O projects had to draw on the existing body of research rather than on new information which was due to be developed as part of FtRG.

Achievement of the programme objectives

The short-term objectives of CFF to identify, trial and encourage the adoption of practices to reduce greenhouse gas emissions, adapt to a more variable climate and increase soil carbon sequestration were realised through the successful delivery of FtRG, AotG and E&O projects.

A substantial amount of information has been developed and communicated as a result of the CFF projects which is expected to have relevance and use for agricultural researchers, extension providers and farmers/landowners. The outcomes of the programmes and projects were found to be variable with the FtRG and E&O programmes perceived as having the greatestsuccess.

Program Legacy

While the evaluation highlighted the realisation of the short-term CFF objectives, it was not possible to understand whether longer-term aspirations to reduce Australian agricultural emissions and enhance productivity and sustainable land use under a changing climate have been realised.It is anticipated that adoption of new practices and technologies (if realised) will occur over a long period, with additional time required before any measurable changes in emissions, productivity or producer resilience can be observed.

A range of outcomes and outputs were identified as being realised through delivery of the CFF programme.The realisation of the CFF legacy, particularly through the use of data and materials generated through the programme, is expected to result in the greatest impact of CFF. This includes a significant number of published peer reviewed papers in scientific journals.A number of risks are associated with the realisation of this legacywhich will need to be carefully managed (including awareness of, and access to, the CFF information).

Lessons learnt from the evaluation of the CFF Programme

The evaluation identified a number of lessons learnt through the delivery and administration of the CFF programmes. Key lessons learnt are summarised below with a full list included in section 7.3 of this report. Lessons learnt through CFF should be used to inform the design, implementation and management of similar large-scale grant programmes.

Key lessons learnt from the evaluation of the Carbon Farming Futures programme include:

mechanisms that support the collaboration between grantees can assist in the facilitation of good project outcomes. Formal coordinationwithin FtRG and the establishment of inter-project conferences for E&Owas found to be an appropriate way to generate and promote collaboration. The design of FtRG and E&O was well received by grantees and industry and should be considered as part of future grants programmes, incorporating these approaches to collaboration.
four years was an insufficient time to measure changes in soil carbon, impacting on the success of FtRG and AotG projects which focused on this research priority area. Unless projects are able to leverage existing sites for trials and demonstrations, it is expected that up to ten years would be required to measure changes in soil carbon as part of future programmes.
the focus of individual projects should balance the needs of the department, grantees and the end-user. Through CFF, particularly E&O projects, grantees wanted to include adaptation and productivity messaging as part of communication and outreach events. This messaging was seen as more closely aligned with the needs and interests of the end-users (such as farmers and landowners), leading to greater attendance at events and engagement with the project.Where a grant programme is seeking behavioural change, the messaging should be aligned with the needs of the end-user to promoteattendance, engagement and adoption.
expectations and requirements of grantees and the department should be clearly articulated as part of large grant programmes, aligning the needs of both parties. As part of CFF, E&O grantees noted some uncertainty around expectations of them during the projects. The requirement to gain approval for all project materials prior to use was, in their minds, an unanticipated burden which resulted in delays to some project activities. Similarly, the level of detail required in project reporting (notably a requirement to attach all materials) was seen as a burden, with a lack of clarity around expectations for these reports.

Despite the concerns of grantees, it is noted that the review process was crucial to ensuring the quality and accuracy of some project outputs.

due to the nature of agricultural projects, grant administrators must ensure there is flexibility in project management to respond to unexpected events. A number of unanticipated challenges were experienced as part of the CFF projects which had the potential to impact on the overall success and delivery of projects. This included weather variations and other seasonal events. The department’s flexible approach to the management of these events was well received by grantees and should be considered as part of the management of future programmes.
where a particular legacy is sought from a grant programme, processes and procedures should be developed to ensure information is appropriately captured and communicated. CFF has the potential to realise a substantial legacy through the use of data and information generated and communicated across the three programmes. The legacy will be at risk if appropriatesteps are not taken to communicate and maintain information and materials, ensuring accessibility of information into the future.

2Introduction

Carbon Farming Futures (CFF), managed by the Department of Agriculture and Water Resources (the department), aims to ‘ensure that advances in emissions reduction technologies and techniques will continue the evolution of management practices in the land sector towards emissions reduction and improved productivity’[2].This aim was later broadened to include identifying and encouraging the adoption of practices toassist farmers toadapt to a more variable climate.

CFF was delivered through three competitive grant programmes; Filling the Research Gap (FtRG), Action on the Ground (AotG) and Extension and Outreach (E&O) delivered between 2012 and 2017 under which 200 projects were funded.

An overview of the design and focus of CFF is included in section 3.2.

Grosvenor Management Consulting (Grosvenor) was engaged by the department to conduct an evaluation of CFF and its three programmes. This evaluation was undertaken at the end of programme and was conducted in four stages, incorporating:

Phase 1: Development of the Evaluation Work Plan and confirmation of evaluation approach
Phase 2: Evaluation of projects funded under round one of the three CFF grant programmes (including E&O projects completed prior to 31 December 2015[3])
Phase 3: Evaluation of projects funded under round two of the three CFF grant programmes (including E&O projects completed after 31 December 2015)
Phase 4: Overarching evaluation of CFF (incorporating analysis from both funding rounds).

This report relates to stage 4 of the CFF evaluation and presents findings and conclusions about CFF as a whole.

3Carbon Farming Futures

3.1Background

The Australian Government, in collaboration with the Department of Industry, Innovation and Science has developed a set of Science and Research Priorities which are designed to increase investment and research in areas deemed to be of critical importance to Australia. Environmental Changeis one of the Science and Research Priorities, which identified three practical challenges:

improved accuracy and precision in predicting and measuring the impact of environmental changes caused by climate and local factors
resilient urban, rural and regional infrastructure
options for responding and adapting to the impacts of environmental change on biological systems, urban and rural communities and industry[4].

The priorities identified opportunities to overcome research weaknesses and utilise research strengths using an interdisciplinary approach to Research and Development (R&D) including:

strategically coordinating long-term research investment in environmental change
encouraging robust international partnerships involving data sharing and researcher exchanges
supporting human resources to run, manage and utilise research infrastructure.

The Australian Government policy has identified the importance of reducing Australia’s contribution to climate change. The priorities were developed to guide research investment and align research withindustry.

Additionally, the ‘Securing a Clean Energy Future’ plan, released in July 2011, sought to align the research and industry focus to carbon price, innovation in renewable energy, energy efficiency and action on the land, in an effort to cut carbon pollution and drive investment in new clean energy sources.

CFF, delivered in support of the ‘Securing a Clean Energy Future’ plan and in alignment with the Science and Research Priorities, seeks to mitigate the contribution of Australian agricultural practices to climate change, whilst maintaining (or increasing) the productivity of farms.

3.2Carbon Farming Futures Design

CFF was designed to contribute to Government policies and plans to reduce the Australian contribution to climate change by:

addressing gaps in current greenhouse gas mitigation and carbon sequestration knowledge
identifying, trialling and encouraging the adoption of technologies and practices which can reduce and manage emissions from Australian farming and agricultural activities.

CFF consists of three individual grant programmes, which each contribute to the programme’s overall objectives:

Filling the Research Gap, which aims to fill known gaps in research relating to abatement technologies and practices that may reduce greenhouse gas emissions or sequester carbon in soils and vegetation[5].
Action on the Ground, which aims to understand the effectiveness and productivity impacts of emerging technologies and practices to reduce greenhouse gas emissions and increase carbon sequestered in soils and vegetation through on-farm trials and demonstrations. The programme also seeks to demonstrate these practices and technologies to farmers and land managers in order to increase their understanding, confidence and willingness to adopt[6].
Extension and Outreach, which aims to increase knowledge, skills, engagement and adoption within the agricultural sector of practices and technologies which can reduce greenhouse gas emissions or sequester carbon in soils and vegetation.

Each of the three grants programmes aim to contribute to the overall CFF aspirations to:

reduce greenhouse gas emissions from the farming sector
increase the sequestration of carbon in soils and vegetation.

Despite being designed around mitigation, the focus of CFF and each programme was expanded to consider adaptation and how to increase the land sector’s resilience to climate change. The long-term productivity impacts of practices and technologies considered as part of the three programmes were also addressed by grantees. As such, the three programmes ultimatelysought to develop, trial and promote practices and technologies that met the CFF aims while maintaining, or improving, productivity and increasing the land sector’s resilience to climate change.

CFF is also closely linked to, and encourages participation in other Government programmes and policies which aim to help reduce Australia’s contribution to climate change. This includes the Carbon Farming Initiative (CFI) and the Emissions Reduction Fund (ERF).

The three CFF programmes (FtRG, AotG and E&O) were designed and managed as discrete competitive grants programmes. While the three programmes all form part of CFF, each grant programme (including the application and assessment processes) was separately managed.

In addition to developing new practices and technologies, the three programmes built upon and utilised existing research and policies relating to climate change and abatement technologies. This included the outcomes of the Australian Government’s Climate Change Research Program (CCRP)[7], conducted between July 2008 and June 2012.

Climate Change Research Program[8]
The Climate Change Research Program (CCRP) was an Australian Government research program which aimed to understand and respond to challenges posed to the productivity of Australian agriculture by climate change.
A total of 50 research and demonstration projects were funded under the CCRP which focused on ‘technologies and farm practices to help reduce greenhouse gas emissions, improve soil management and assist the agricultural and fisheries sectors with options to adapt to a changing climate’. Funding under CCRP was allocated across the following programs:

Reducing Emissions from Livestock Research Program
Nitrous Oxide Research Program
Soil Carbon Research Program
National Biochar Initiative
Adaptation Research Program
Demonstration on-farm or by food processors.

It is reported that the CCRP has increased the understanding of the sources of greenhouse gas emissions and revealed potential methodologies to reduce emissions and sequester carbon in the landscape, supporting new ways for the agricultural sector to manage and reduce emissions while maintaining productivity.
Outcomes of the CCRP have been utilised to develop carbon offset methodologies approved under the Carbon Farming Initiative and Emissions Reduction Fund.

The original design of the programme intended that:

three discrete funding rounds would be conducted for FtRG and AotG
there would be no discrete funding rounds for the E&O, enabling applications to be submitted at any time. Applications were to be assessed in bulk at defined dates throughout the programme
practices and technologies explored as part of CFF projects would evolve and progress through multiple rounds across the three grant programmes. This would allow research developed as part of FtRG to be subsequently trialled and demonstrated by AotG and ultimately extended through E&O, building upon and utilising existing knowledge (as seen in Figure 1)
the process would continuously build and develop upon existing resources and knowledge.

Priorities for each funding round/assessment period are outlined in the grant programme guidelines[9], and were determined by comparing current gaps to the objectives outlined by the overarching strategy[10].