Final Executive Report

Evaluation of the Jobs Fund Heritage Program

FOR

The Department of Sustainability, Environment, Water, Population and Communities

31 August 2011

SECTION 1 INTRODUCTION

Purpose and structure of this report

This Executive Report serves the purpose of an ‘extended’ executive summary for Courage Partners’ evaluation of the Jobs Fund Heritage Program. It provides summaries of findings (with brief discussion) and recommendations. It is intended for use as a briefing aid to senior Department staff and the Minister.

A more extensive Report (but without an executive summary) is provided separately, which sets out Courage Partners’ more detailed discussion and analysis, key findings and recommendations.

The following is structured around the evaluation’s three Terms of Reference (TOR), with a summary of recommendations presented in Section 3. Please note that project examples are used throughout this document to illustrate findings or demonstrate better practice. A brief description of each project involved in the evaluation sample is provided in Attachment A of the main report.

Background to the Jobs Fund Heritage Program

The Jobs Fund

The Jobs Fund was a $650 million Australian Government initiative designed to support and create jobs and skill development through projects that built community infrastructure and social capital in local communities. The Jobs Fund was a discretionary grants program which commenced in June 2009 and finished in June 2011.

The Jobs Fund Heritage Program

As part of the Jobs Fund, $60 million was allocated for the Jobs Fund Heritage Program (or the Program). To be considered for funding under this program, projects also had to deliver positive heritage outcomes. The Program focused on strengthening the role of heritage in the community by protecting, conserving and promoting: National Heritage-listed places; National Trust properties; Community heritage places; and Natural heritage places.

As well as seeking to deliver positive heritage benefits, the Program aimed to provide employment and economic stimulus, particularly in areas of high unemployment. Program funding was also intended to improve heritage infrastructure and bring increased value and opportunities to heritage places.

Assessment and recommendations for projects were undertaken in line with heritage specific criteria. Approximately 571 project applications for heritage grant funding were received for the Heritage Program across two funding rounds in 2008-09 and 2009-10. Of these, 191 were accepted and funded over the two rounds. Projects were dispersed across Australia with grant sizes ranging from $20,000 to $2 million. Issues that were also taken into consideration included whether the project would contribute to one or more of the priority heritage themes of: Australian Democracy; Peopling a Nation; and Diversity of Landscapes.

Purpose of the evaluation

The purpose of this evaluation was to provide the Department with a source of information concerning:

  • The effectiveness of the Jobs Fund Heritage Program in delivering the heritage outcomes detailed in the heritage guidelines for the Jobs Fund Program;
  • The social and economic impacts (including potential impacts) of heritage grants (funded and non-funded) on the general community, as well as level of unmet demand; and
  • Opportunities for improving and widening future Australian Government involvement in heritage initiatives.

The evaluation was based on a review of a sample of 33 selected projects (33 file reviews with 14 of these including site visits) as well as interviews with a sample of ten unsuccessful applicants. It will be used to support reporting on the performance of the Jobs Fund Heritage Program, as well as the design and development of future heritage programs and associated policy proposals.

Scope and focus

The evaluation was focused at the Program level, on an examination of the effectiveness and impact of heritage grants on communities, including further opportunities for Government involvement in heritage. As there was limited aggregated Program data available from the Department, the evaluation approach was based on the review of a sample of funded and unfunded projects to provide indicative results and judgments about Program level performance, impacts and opportunities. As requested by the Department, detailed data for this evaluation was collected via site visits to a sample of project sites.

Please note that this evaluation did not include:

  • An examination of jobs creation data or the immediate impact of heritage grants on direct employment from initial program funds (although any subsequent indirect employment impacts have been explored where possible). Courage Partners understood that job creation data and direct employment impacts of the Heritage Program were being examined in a separate evaluation by Sweeney and Associates;
  • An examination of the efficiency or effectiveness of the administration of the Heritage Program, as these aspects were being addressed by the concurrent ANAO performance audit;
  • The assessment and selection process for successful project applications.

Methodology and Sources of Data Collection

Key methods and sources of data collection

As agreed by the Department, the evaluation focused on the collection of essentially qualitative data as follows:

  • Interviews with key Departmental staff responsible for administering the Program;
  • Desk-top review of a sample of 33 completed1 project grant applications and project reports;
  • Desk-top review of, and follow up telephone interviews with, a sample of 10 unsuccessful project grant applicants;
  • Follow up face to face site visits with a cross section of 14 completed funded projects (including one pilot visit to test data collection instruments and to inform planning)2. Site visits involved an in-depth interview with the grant recipient, interviews with other key project stakeholders and (normally) inspection of the heritage property and the works completed.

Terms of Reference and Key Evaluation Questions

The following TOR was developed to address the purpose of the evaluation. The Key Evaluation Questions detailed below each TOR guided the research and data collection.

TOR 1: To what extent has the Jobs Fund Heritage Program been successful in meeting the heritage outcomes as detailed in the heritage guidelines for the Jobs Fund Program?

1.To what extent did selected projects contribute to the heritage values of a place, including conservation, protection, adaption or interpretation works?

2.To what extent were selected projects consistent with the principles of the Burra Charter, management plans currently in place, and state, territory and Commonwealth legislation?

3.To what extent did selected projects contribute to one or more of the priority heritage themes identified by the Australian Heritage Council, and include elements designed to promote awareness of the activity, and of the social and economic value of the heritage property?

4.To what extent did selected projects provide on-going direct and indirect social and economic benefits in the community (such as increasing tourism, or increasing the appropriate utilisation/value/rental returns of heritage properties)?

TOR 2 - What has been the impact of heritage grants on the general community, both economically and socially?

1.What are the social and economic impacts and benefits (including potential benefits) of selected heritage grants projects on the general community?

2.Was the grant useful in leveraging additional funding support and community recognition of the project?

3.What are selected non-successful grant applicants’ views on:

  1. the social and economic impact of not receiving heritage funding on the general community?; and
  2. the impact of not receiving heritage funding on the status of the heritage place?

4.What are key project stakeholders’ opinions of the level of unmet demand for further heritage funding in their community? (For both successful and unsuccessful applicants).

TOR 3: Are there further opportunities for improving and widening future Government involvement in heritage at the local and state community level?

1.What are stakeholders’ views of opportunities for improving and widening future government involvement in heritage?

1 Completed projects were those that had submitted a draft final or final report that has been accepted by the Department.

2 Projects selected for site visits were drawn from the sample of 33 projects selected for desk-top review.

SECTION 2 FINDINGS (AND BRIEF DISCUSSION)

TOR 1: TO WHAT EXTENT HAS THE JOBS FUND HERITAGE PROGRAM BEEN SUCCESSFUL IN MEETING THE HERITAGE OUTCOMES AS DETAILED IN THE HERITAGE GUIDELINES FOR THE JOBS FUND PROGRAM?

Achievement of intended outcomes

Finding 1: The Jobs Fund Heritage Program has been successful

Based on the selected 33 projects identified for the evaluation, the evaluation team found that the Program has been successful in meeting the heritage outcomes detailed in the heritage guidelines for the Jobs Fund Program. One exception to this was in projects’ use of supporting promotional activities.

In particular:

  • All works identified in the original applications for the selected projects were undertaken and completed (including agreed changes of scope) by the time of reporting.
  • All projects reviewed had contributed to the recognised heritage values of a place (which included conservation, protection or interpretation). Most projects directly contributed to those heritage values, and some indirectly supported heritage outcomes by improving access to the heritage site.
  • All selected projects indicated that their plans were consistent with the Burra Charter and Management Plans and this was attested to by an expert or technical adviser in most cases.
  • All projects contributed in a range of ways to one or more of the heritage themes: A Free and Fair Australia; Peopling a Nation; and Diversity of Landscapes. About half (17 out of 33) of the projects were reported as consistent with a multiple of these themes.

Promotions and awareness raising

Finding 2: There was inconsistent completion of project promotional activities

On the whole, the file review revealed that there was not a comprehensive approach taken by grant recipients to the design and delivery of promotional activities around projects. The evaluation team concluded that overall, promotional activities were not given sufficient priority by grant recipients in project management and delivery.

The evaluation team found that:

  • Information in applications about strategies for promotions was generally limited.
  • Promotional activities were completed as per strategies set out in their applications in about 45% of works (15 out of 33).
  • In terms of implementation, many plans for promotional activities had either been delayed, or not pursued with vigour. The team concluded that there was insufficient promotion and awareness raising, due to a variety of reasons, including: lack of time; lack of skill or lack of funding.
  • Better practice examples found on site visits showed that some sites had comprehensive marketing plans in place, and were actively involved in seeking out promotional and marketing opportunities both locally and more broadly.
Finding 3: Promotions and awareness raising is important in maximising project benefits

Better practice examples in site visits evidenced that strong promotions can maximise positive heritage, social and economic impacts for the project and the community.

Social and economic impacts

Finding 4: The program achieved a number of social and economic impacts, with further potential benefits identified

The Jobs Fund Heritage Program has resulted in a number of social and economic benefits for communities. However, site visits did highlight that it is simply too early to gauge the extent of full impacts; which will also depend on the strength of promotional aspects.

All project final reports examined indicated progress towards achieving desired social and economic impacts. Site visits enabled the information in these reports to be confirmed and extended. However, site visits did highlight that it is simply too early to gauge the extent of full impacts; which will also depend on the promotional aspects outlined earlier.

Based on the sites visited, the evaluation team identified that the potential identified for social and economic impacts was significant, and this is discussed further under TOR 2.

The evaluation team further identified that:

  • Project reports provided limited information regarding the full extent of social and economic impacts achieved for each project. However, site visits did provide some very useful examples of levels of improvement and indicators of potential benefits.
  • There were a couple of sites visited, where the team concluded that potential impacts may not be realised in the short term, due to a lack of skill in promotions, lack of data being collected and lack of clarity around objectives for the site, for example around increased access or visitation.
  • On the whole, projects did not maintain robust measures around social and economic impacts, either before or after project works and this is an area to be addressed by the Department for future grants programs. The evaluation team observed that part of the reason for this is the scarcity of clear measures around performance of heritage projects, both generally and for this specific program.

TOR 2: WHAT HAS BEEN THE IMPACT OF THE HERITAGE GRANTS ON THE GENERAL COMMUNITY, BOTH ECONOMICALLY AND SOCIALLY?

Social and economic benefits

Finding 5: Potential for achieving social and economic benefits was identified

Social and economic benefits varied, depending on the nature of the project; the amount invested; and the timeframes required for impacts to be realised. Due to the recent conclusion of the projects; many of these can only be identified as potential impacts.

In 12 of the 14 sites visited, the evaluation team found strong indications that potential impacts were likely to come to fruition in the short term.

However, as mentioned under TOR 1, the means by which social and economic impacts could be measured was limited; with little information on for example, increase in visitor numbers not just to the site but also to the local community or region. Further details regarding tourism information, visitor experiences, income and expenditure before and after project commencement was rare. There were no examples of community perception data regarding social or economic impacts. To better manage their sites and prepare for potential projects, heritage sites should consider collecting a range of useful baseline data (for example, visitor numbers, running costs and satisfaction levels) that would enable them to better measure the impacts of any future projects they get involved in, or just better managing their site.

What was apparent in all sites visited was the huge depth of passion and commitment from project stakeholders about the projects, and heritage more generally. This extended from the volunteer groups in small communities to larger heritage organisations such as the National Trust and through to state departments and local councils. The evaluation team found that the Jobs Fund Heritage Program had effectively engaged with a genuine community interest in heritage, and had raised the profile of the Department with stakeholders and their communities.

Nature of social and economic impacts identified

Finding 6: A variety of social and economic impacts were identified

During site visits, a number of economic and social impacts were identified as resulting from the Jobs Fund Heritage Program:

  • Community pride;
  • Enhanced visitor experience;
  • Extending the life of the site;
  • Increased time of stay in the area;
  • Improved aesthetics;
  • Increased access;
  • Increased capacity for further use (adaptive re-use);
  • Applied training opportunities and preservation of heritage skills; and
  • Other short term impacts from project work (other than job creation).

These are described with corresponding project examples in the following.

1.Community pride

The large majority of stakeholders interviewed on site visits found it easy to articulate numerous examples of how the project had contributed to a stronger sense of community pride and understanding of their heritage and culture. This was apparent in most, and identified by stakeholders in 12 out of 14 visits.

To illustrate:

  • At The Roxy – stakeholders were thrilled with the extent of community engagement and interest in the project; which extended from local Bingara residents to the wider Australian Greek community;
  • The Brickenden / Woolmers sites have engaged the local Longford Community and more broadly highlighted Tasmania’s role in assignment of convicts both nationally and internationally;
  • At Hermannsburg, with the refurbishment of the precinct’s dining room and bakery, the community has become increasingly supportive of the site’s heritage value and what it now has to offer them and people who visit. It was reported by stakeholders interviewed at the site that it is now proudly valued by the local community.

A number of staff at the Department commented on the level of commitment from project stakeholders across the various grant categories, and made the point that community engagement was vital to the success of the project.