Appendix 4: Technical report

Developing models to estimate the occurrence in the English countryside of Great Crested Newts, a protected species under the Habitats Directive [WC1108]

End-user testing – to test model applications and fitness for purpose

Summary

The views, knowledge, understanding and attitudes of potential end-users towards predictive modelling were assessed using three approaches: (1) an online questionnaire; (2) a workshop for a wide range of stakeholders attending the Herpetofauna Workers’ Meeting; and (3) a workshop for professionals working in local and national government agencies, consultants, developers and utility providers. The vast majority of end-users were receptive to the idea of predictive modelling being used in decision-making for great crested newts, although there were some qualifiers and reservations.The main areas identified as having potential for applying SDMs were status assessment;risk assessment;impact assessment; assessing connectivity; informing planning and licensing decisions; targeting surveys; improving the outcomes and cost-effectiveness of surveys; habitat creation and management; and development mitigation.Whilst the general concept of predictive modelling is widely understood by end-users, there is a need to raise awareness of its value in particular applications and the attendant limitations.Concern was expressed over the reliability of model outputs and predictions, and whether the levels of confidence associated with predictions were appropriate for due-diligence and decision-making. Indeed, at a site-specific level modelling is no substitute for on-the-ground surveys. Implementing a modelling protocol would clearly require widespread training and capacity building, and the uptake may be easier for large organisations with the relevant infrastructure already in place. Considerable concerns were expressed over existing data management and data flow systems, which – in their current state of functionality – would not be able to efficiently provide the data that models require in terms of quality or quantity. Indeed, there was a risk that both input data and model outputs could be misused, both intentionally and unintentionally. There was no consensus over whether modelling might be best implemented through a local, regional or a centralised national system. Whatever options are proposed for modelling, there is a need for governance to be clear and authoritative, with common standards. Changes to legislation, policy and guidance may go some way to achieving this.

4.1.Introduction

Incorporating predictive modelling into great crested newt conservation requires consultation with potential end-users and the wider conservation community. This would be needed with any significant amendment to policy or practice, but – depending on the precise applications and implementation mechanisms –it will be particularly important for modelling since it could involve a fundamental change to existing practice. Moreover, there might be a need for substantial shifts in technical competence, procedure, regulation, and the practice of key routine activities including newt survey and mitigation.

We investigated end-user and conservation community needs via three main approaches:

  • An online questionnaire of selected key potential end-users (January - February 2015);
  • Alarge workshop at a conference for a range of practitioners and volunteers in the amphibian conservation community (the Herpetofauna Workers’ Meeting, Newcastle, February 2015);
  • A targeted workshop for representatives from key end-user sectors (London, June 2015).

The online questionnaire sought views on broad issues, such as the degree to which those involved in great crested newt decision-making are aware of modelling, and understanding of its potential benefits.We also asked detailed questions about the specific needs of users. The online questionnaire also sought expressions of interest in further participation in the project, as well as requesting data and case studies. In the Herpetofauna Workers’ Meeting, we aimed to assess the views of a wider cross-section of potential users of modelling, as well as those who would not use modelling but are still involved in great crested newt conservation. Finally, the London workshop aimed to explore the views of a focused group of potential end-users from different professional sectors, based on interim results obtained up to the end of June. This allowed a more thorough assessment of some key issues, and also allowed us to specifically recruit construction industry representatives, since this sector was slightly under-represented in the two earlier consultation exercises.

In addition to these three specific exercises, we also gauged the responses and needs of potential end-users via less structured means. These included one-to-one meetings between the DICE modelling team and end-users in July/August 2015; ongoing liaison with individuals who demonstrated particular interest from the formal consultations; and discussions at the Natural England Great crested newt Pilot Advisory Group meetings.

When undertaking consultations we emphasised that the contract was to explore the issues, and not define a pre-determined policy shift.

4.2.Methods

4.2.1. Online questionnaire

The questionnaire was designed to address specific issues regarding the requirements of potential end users. An initial set of questions was drawn up and circulated within the project team, DICE staff involved with questionnaire design, and the project steering group for refinement. Following this a draft version of the questionnaire was tested using the online platform SurveyMonkey( This was then piloted within the project team and then further refined before being distributed to the consultees.

The final version of the online questionnaire consisted of 11 pages (see Annex 1 for structure).

Most responses are shown as simple counts of multiple choice answers with percentages. We allowed an “other” option with free text where we considered it likely consultees might have responses in addition to our standard answers. For questions E2, E3 and F1 the consultees were asked to rate the importance of certain issues; for these questions we give an average rating, which is the average of the scores from 1 (not important) to 5 (very important).

The number of initial consultees for the questionnaire was limited to 24. Although the intention was to send the questionnaire directly to a larger group of consultees, the project steering group advised that this was not permissible within government survey protocols without significant additional investigation, which would have compromised the timing of the project. The 24 consultees were selected from a shortlist of potential respondents whose job, at least in part, involves making decisions related to great crested newts. The final list of consultees was selected in order to include as wide a range of sectors as possible. The 24 initial consultees were drawn from the following sectors:

  • Local Planning Authorities and associated contacts: 6 consultees
  • Licensing authorities, statutory conservation advisors and other regulators: 6 consultees
  • Construction industry: 2 consultees
  • Utilities and major infrastructure providers: 2 consultees
  • Ecological consultants: 3 consultees
  • Data collectors and data providers: 2 consultees
  • NGOs and civil society: 3 consultees

Each consultee was sent an email invitation to complete the questionnaire containing a brief introduction to the project, the background to the questionnaire and a link to the SurveyMonkey questionnaire. Consultees were encouraged to forward the link to colleagues and contacts. The consultation started took place from January to February 2015.

4.2.2.Herpetofauna Workers’ Meeting workshop

The objective of the workshop was to assess how a modelling approach to great crested newt conservation, might be received by a wider stakeholder group. We chose the Herpetofauna Workers’ Meeting as a forum for this assessment, as it has a diverse attendance, and because workshops are a traditional feature of the programme. Delegates at the meeting are representative of the typical stakeholders likely to be interested in modelling, including scientists, consultants, statutory agency staff, volunteers and NGO representatives. They therefore represent potential end-users as well as others with roles closely linked to great crested newt decision-making and data collection.

The workshop was advertised in advance as part of the promotion for the Herpetofauna Workers’ Meeting. Those registering for the meeting were given the choice of opting for the workshop, from a choice of four. The meeting took place on 7-8 February 2015 at the Vermont Hotel, Newcastle-upon-Tyne, and was organised by Amphibian and Reptile Conservation (hereafter, ARC) in conjunction with Amphibian and Reptile Groups UK (hereafter, ARG UK). The programme includes a variety of talks on amphibian and reptile conservation as well as a choice of participatory workshops.

Jim Foster (ARC) and Richard Griffiths (DICE) led on preparing the workshop format and content, with input from Dimitrios Bormpoudakis (DICE) and Thomas Starnes (ARC), and all four delivered the workshop. Liam Russell (ARC) and Jim Foster led on the post-workshop reporting.

Participants signing up for the Herpetofauna Workers’ Meeting were asked to select two of four workshops run during the event. The information provided to potential registrants included the following description of the great crested newt modelling workshop:

Workshop A: Towards a risk-based approach for great crested newt conservation?

Leaders: Richard Griffiths (DICE) & Jim Foster (ARC)

The locations of most great crested newt ponds is unknown, making for complex decision-making. DICE, ARC and the Freshwater Habitats Trust are collaborating on a Defra-funded project to scope a new approach to conserving newts. We’re developing models to predict where newts occur, and assessing risk based decision-making. Underlying this approach is a focus on conserving populations rather than individuals. In this workshop we’re seeking practitioner reaction to our ideas. How would this new approach impact on surveys for development, mitigation planning, or pond creation schemes? Can you suggest any improvements to our proposals? ARG members, local authority staff and consultants are especially encouraged to participate.

One hundred and seven out of a total of 169 registrants signed up to participate within the workshop. It was the most popular workshop of the four offered at the conference.

Two workshop sessions were held:

  • Session 1: Saturday 7 February 2015 (51 registered participants)
  • Session 2: Sunday 8 February 2015 (56 registered participants)

Both sessions were led by Richard Griffiths (DICE), Jim Foster (ARC), Thomas Starnes (ARC) and Dimitrios Bormpoudakis (DICE). The sessions lasted 1 hour 30 minutes each.

The workshop started with a brief introduction to species distribution modelling, and an exploration of how this could be applied to great crested newts. We then briefly demonstrated two potential practical applications of modelling:

(1) Optimising survey effort when assessing the impacts to a hypothetical pipeline development, and

(2) A case study of modelling being used to inform land use allocation by a Local Planning Authority (based on an ARC project with Wrexham County Borough Council).

Following this introduction the participants were asked to form small groups (with approximately 7-12 members) to discuss the value of using predictive models for great crested newts in one of five scenarios (approximately 40 minutes, during which time the workshop leaders circulated to assist with queries arising from the discussions):

  • Scenario 1: Modelling as a tool in impacts assessment and mitigation planning
  • Scenario 2: Modelling as a tool in survey planning
  • Scenario 3: Modelling as a tool in Local Planning Authority development control (determining planning applications)
  • Scenario 4: Modelling as a tool in Local Planning Authority strategic planning (allocating future land use)
  • Scenario 5: Modelling as a tool in planning habitat creation & management

For each scenario, participants were asked to consider the following points and record their written comments in a table on a prepared reporting sheet:

  • Advantages: What are the potential benefits of modelling for this application?
  • Disadvantages: What are the possible downsides or risks of modelling for this application?
  • Barriers to implementation: What factors would make it more difficult for modelling to be used in practice?
  • Potential users: which organisation(s) do you think might use models for this application? (please tick & add more if desired)

NE/NRW/SNH

Local Planning Authority

Ecological Consultant

Developer

University

Landowner

Local Records Centre

Volunteer surveyor

ARG

National NGO (e.g. ARC)

Local NGO (e.g. Wildlife Trust)

Other (please state):

  • General comments

After the small groups had discussed their responses to the questions, a spokesperson from each group presented the outcome of their discussion to the whole workshop (approximately 20 minutes). Given timing constraints, the workshop leaders did not respond in detail to individual comments made during the feedback sessions. The reporting sheets were collected for later transcription.

Towards the end of the workshop, we gave a brief summary of the next stages for the project, asked participants if they had any datasets we might use in model development, thanked them for their time, and noted that a project report would be due in autumn. We stressed that the modelling and risk-based approaches under discussion were still being explored, and that government had not yet decided that this would be pursued. We explained that the workshops they had helped with were part of that exploration.

We ended the workshop with a “straw poll” by asking participants the following question: “Would you eventually like to see predictive modelling included as a fundamental aspect of great crested newt conservation policy, planning and guidance?”, with possible responses: Yes, No, Don’t know.

4.2.3. Targeted end-user workshop

Invitations to this workshop were based on liaison with the Steering Group, the Natural England-led Great crested newt Pilot Advisory Group, feedback from the online questionnaire, and discussions held at the Herpetofauna Workers’ Meeting workshop. We invited 59 representatives from the following sectors:

  • Developers/Utilities companies
  • Ecological consultants
  • Government agencies
  • Local planning authorities (LPAs)
  • NGOs and recording groups.

The workshop arrangements were agreed with the Steering Group in advance, and invitations agreed by email with a maximum of 25. The workshop was held at Nobel House, Defra, in central London, from 11.00 to 16.00 on 30 June 2015.

The workshop structure was partly based on our successful experience with the workshop at the Herpetofauna Workers’ Meeting (see above). The first section set the scene and provided background to modelling and preliminary results obtained so far. This workshop, however, was designed to elicit slightly different information given that we were able to select a more targeted audience, and we had interim results from the modelling exercises. We discussed the interim results of the modelling work done using environmental DNA (eDNA) surveys in Kent, Cheshire and Lincolnshire. We demonstrated the different model types, discussed model performance in different regions and at different scales, and briefly explained the data, hardware, software and expertise requirements. This was followed by a general discussion.

We then divided the participants into six break-out groups by sectors. One member of the contract team facilitated each break-out group for a 45 minute period. During this facilitated discussion, we asked each group to discuss and report on the following using astructured proforma:

  • Applications (Please list possible applications of a modelling approach within your sector)
  • Advantages (What are the benefits of modelling for this application?)
  • Disadvantages (What are the possible downsides or risks of modelling for this application?)
  • Implementation (What might be needed to implement modelling for this application (e.g. training, funding, hardware etc)?)
  • General comments.

One representative from each group provided oral feedback at the end of the session. Groups were also asked to note any related issues that arose during the course of their discussions on post-it notes, and these were collated at the end of the workshop. Each sector was also asked to provide a response to the question: “Does your group think that a modelling approach could help decision-making within your sector?” with three possible responses: Yes, No, Don’t know.

Following this we held a discussion on workshop conclusions, and finally a brief section in which we explained the next steps in the project, with an offer for one-to-one modelling meetings and a request for case studies.

4.3. Results

4.3.1.Online questionnaire

A total of 41 responses was received, four of which were mostly blank and therefore removed from the analysis. The remaining 37 responses were included in the analysis.

Questionnaire Section A: User details

Question 1: Name

This was provided by 34/37 respondents (92%)

Question 2: Organisation

This was provided by 33/37 respondents (89%)

Question 3: Position

This was provided by 33/37 respondents (89%)

Question 4: E-mail

This was provided by 34/37 respondents (92%)

Question 5: Contact telephone number

This was provided by 31/37 respondents (84%)

Question 6: What sector do you work in?

This was answered by 37/37 respondents (100%)

Table 1. The different sectors represented at the workshop.

Sector / Response count / Percentage
Local Authority / 12 / 32.4
Statutory Agency / 15 / 40.5
Development (including construction, infrastructure, utilities, etc) / 4 / 10.8
Ecological/Environmental Consultancy / 3 / 8.1
NGO / 3 / 8.1
Other (please specify) / 0 / 0

Question 7: How much of your job is concerned with decisions relating to great crested newts?

This was answered by 37/37 respondents (100%)

Table 2.Time spent dealing with decisions relating to great crested newts.

Sector / Response count / Percentage
Not at all / 2 / 5.4
About 25% / 25 / 67.6
About 50% / 4 / 10.8
About 75% / 5 / 13.5
All of my job / 1 / 2.7

Questionnaire section B: Why do you need data on great crested newts?

Question 1:Why do you need information on great crested newts to fulfil your job responsibilities? Pleasetick all that apply