in Public Life in Scotland
APPLICANT RESEARCH 2016
October 2017
1
Commissioner for Ethical Standards
in Public Life in Scotland
Contents
Introduction
Executive Summary
Key Findings and Recommendations
Survey objectives
Survey methodology
Approach
Changes to the timing and frequency of sending out surveys
Changes to the questions
Sample size and response rate
Demographic analysis
The survey results in detail:
FIRST TIME APPLICANTS AND EXPERIENCES OF THOSE WHO HAVE PREVIOUSLY APPLIED
MOTIVATION
HOW APPLICANTS FIRST FOUND OUT ABOUT THE VACANCIES
ADVERTISING AND APPLICATION
FINAL ASSESSMENT AND INTERVIEW
FEEDBACK
OVERALL VIEWS ON THE PROCESS
In conclusion:
Introduction
This report presents the findings of surveys carried out during 2016of applicantsfor public appointments in Scotland.
We carried out annual surveys in 2012, 2013 and 2014 to help us to understand whether changes being made to the way that the appointments system was run were having an impact on the way that applicants viewed it. Following the publication of our 2014 report on the annual survey, we engaged in discussions with the Scottish Government about the usefulness of running annual surveys. We agreed that a change in approach whereby we contacted applicants sooner after they applied might well lead to higher response rates. We also felt that running individual surveys for each appointment round held would allow us to ask more tailored questions. As a consequence, from May 2016,we commenced running surveys on a round by round basis, shortly after completion of each round. This report highlights the main findings to “core” questions asked in all the surveys conducted during 2016 as well as providing some case studies of questions asked in individual rounds. It covers appointment rounds that completed in 2016 so also includes some that started in 2015.
Our change in approach is intended to help us to gather more, and more detailed and informative, applicant views. Both we and the Scottish Government continue to benefit from the views of people on the process and the opportunities that this affords us to learn and to improve Scotland’s public appointments system. The survey methodology and response rates are set out in the report.
1
Commissioner for Ethical Standards
in Public Life in Scotland
Executive Summary
The 2016 report invited a total of 805 applicants across 16 appointment rounds to express their views on the process. 356 applicants chose to take up this offer. This is an approximate 9 percent increase on previous years' surveys.
Of the applicants who provided their views, 45% were applying for a public appointment for the first time and of those, 52% were women and 37% were under 50. 78% of respondents provided demographic data allowing us to understand the views of some groups who are underrepresented across public body boards.
People who had applied previously had a mixed experience of applying again. 68% felt it was the same, 19% felt it was worse and 13% felt it was better.
The main reason that applicants give for applying is that they feel their knowledge, skills and experience are a good fit for the role. People under 50 and BME people are, in comparison with others, more motivated to apply due to the opportunity provided for personal and professional development.
The factors in the publicity that influenced most people’s decisions to apply were that the advert and application pack “sounded like they were looking for people like me”.
23% of respondents were invited to the final stage of assessment. The majority were very positive about two aspects of that stage. 80% felt that the form that the assessment/interview would take was clear. 88% indicated that the interview was conducted well or very well by the panel members. However, only 65% felt that the interview questions reflected the skills, knowledge and experience asked for in the pack.
27% of respondents did not feel that the application process was fair and transparent.
The majority of survey respondents took the opportunity to provide comments on all aspects of the appointment process, including on whether they received feedback and the quality of feedback received.
It is clear from the comments received that many respondents had a very positive experience of applying, whether successful or otherwise whilst other respondents did not. All of the comments received provide invaluable information. Lessons from the applicant surveys run by the Commissioner will continue to be shared with the Scottish Government and selection panels on an ongoing basis. The Scottish Government’s Public Appointments Team has the role of identifying which of the practices that people were positive or negative about could or should be amended in order to ensure improved and improving applicant experiences in future.
Key Findings and Recommendations
As part of our ongoing discussions with the Scottish Government about the applicant survey, we have jointly developed a range of key findings and recommendations. These have been split into 3 areas:
1 – Changes to the survey.
As explained in the introduction and Executive Summary, this report has been constructed on the basis of individual reports run at the conclusion of the appointment round. This has resulted in a far higher response rate than running the survey annually and has provided a better and more reliable range of views than previously. It is therefore our intention to continue to collect results in this way.
Similarly, it is also our intention that the practice of collecting views round by round is further improved by:
1)Endeavouring to send surveys out even more promptly (within a month) after the close of each round, so that information is as fresh in the applicant’s mind as possible; and
2)Continuing to consider module questions which are bespoke to each round, in order to learn more about how different approaches in application and assessment are being perceived.
Finally, it is our intention to amend the core question about whether applicants were able to discuss any part of the application with anyone involved in the recruitment process. This question returned a result of 17% of applicants feeling that they were not able to contact anyone. We therefore intend to amend the question so that that any applicant selecting this option is further probed to understand why they felt unable to contact anyone, given that all application packs contain contact details. It would be very helpful in future to understand why applicants do not feel able to make contact about their application.
2 – General Recommendations
There are 2 key recommendations about improvements to the process for all appointment rounds. These have been made as it is considered that they will help to address a range of the key findings from the report. These are:
1)To ensure that the main attributes required for applicants to be successful in the role are highlighted as prominently as possible in adverts and application packs; and
2)To explain as clearly as possible in application packs, and even more so in invitations to interview, how applicants should expect the second stage assessment process to be conducted, what is being assessed in each stage of this process and why each assessment method is considered most suitable to assess each attribute.
These recommendations seek to address the overall result that just over one quarter of applicants (27%) did not consider the process to be fair and transparent.
The prominent wording in adverts / application packs also seeks to build on the fact that 86.5% of applicants were motivated to apply for the position in the first place as they thought that their skills, knowledge and experience were a good fit for the role, and just over 70% of applicants who were influenced by the advert and / or application form, were influenced due to the fact that the advert / application pack sounded like they were looking for people like them. This should therefore help to ensure that applicants are as clear as possible about what is expected of the role, and therefore should attract a larger population of those who are suited to it.
The clarity of assessment should also help to address the fact that although 88% of applicants considered the interview to have been held well or very well by panel members, a smaller percentage (65%) believed that the questions reflected the skills, knowledge and experience asked for in the application pack.
3 – Panel / Board recommendations
These are a set of recommendations which panels may wish to consider when planning for future appointment rounds. Some are based on aspects of a round which seem to have had a particularly positive impact onan under-represented group and so which may be of interest and help to panels who are keen to attract members of that group in future. Some are more general in nature and could be useful for public body boards to consider in terms of succession planning and general awareness raising of their work or brand for attracting applicants to apply to future rounds.
Younger applicants:
- seem to be more highly motivated by the prospect of the appointment being a good professional development opportunity than other groups. Therefore, emphasising the professional aspects / experience that the role would bring to an applicant’s career history could be useful. Also, the board may wish to consider providing development opportunities such as board apprenticeships or mentoring.
BME applicants:
- seem to be more highly motivated by the prospect of the appointment being a good personal development opportunity than other groups. Therefore, emphasising the personal gain and sense of achievement that undertaking the role provides could be useful. Also the board may wish to consider providing development opportunities such as board apprenticeships or mentoring.
First time applicants:
- are more likely to find out about the appointment opportunity through a professional network or personal contact than other applicants. Therefore, considering spreading word of appointment opportunities via these routes should attract more new applicants. Likewise, public bodies and their boards could consider getting involved with professional networks and / or grassroots organisations at an earlier stage to promote the work of the body and increase brand awareness; and
- are more likely to have been influenced by the advert and application pack than other groups. Therefore, ensuring that the attributes required for the role are emphasised prominently is even more important when wanting to attract more new applicants to the process.
General:
- Welcome letters from the appointing Minister and video messages to highlight appointment opportunities were used in some rounds and received some positive comments about their use;
- Two appointment rounds advertised one position as being “of particular interest to people who wish to develop their skills as part of a Board and are willing to learn”. This wording seems to have been positively received by applicants; and
- The Scottish Children’s Reporter Administration used an interview panel comprised of care experienced young people as part of the assessment process. This tailored approach to as assessment process, by involving those directly affected by the work of the body, seems to have been particularly attractive to applicants.
Survey objectives
The main aims of the survey, which is run on an anonymised basis, are to:
- enable people to share their honest views about the appointments process;
- gain an understanding of the first-hand experiences of people who apply;
- identify apparent barriers and enablers present within the appointments process, particularly where these appear to be significant for any currently; underrepresented group;
- establish any common factors that appear to impact on the progression of such applicants; and
- identify aspects of an appointments process that should be revised or introduced to:
- enhance the applicant experience;
- encourage applications; and
- enable all applicants to apply on an equal footing.
Survey methodology
Approach
The survey was conducted on-line using the accessible survey tool SurveyMonkey™. An on-line approach was considered to be preferable for a number of reasons:
- The cost of printing and posting paper-based survey forms as a matter of course would be prohibitive and wasteful. However, a paper-based copy of each of the surveys was available for anyone who preferred to complete it this way. One respondent during 2016 took up this offer;
- The public appointments application process is predominantly on-line and therefore applicants in the main have internet access; and
- The Scottish Government held an email address list of applicants who had indicated that they would be willing to participate in such research.
Changes to the timing and frequency of sending out surveys
From May 2016 the timing and frequency of sending out the surveys was changed such that surveys were sent out following the end of the round and the public announcement as to who had been appointed. The amount of time following the announcement varied between bodies from within one month and up to 4 months. Usually a number of surveys were sent out at around the same time (between three and five surveys) and they remained open for around one month, with a reminder email being sent to all applicants at the approximate half way point.
Changes to the questions
As well as changing the timing of sending out the survey, we also considered the questions being asked. The previous annual surveys were fairly detailed asking a range of questions of applicants about all aspects of the process. This raised concern that the questionnaire may be too long and complex which could put some people off. We therefore reviewed the “core” questions and reduced these to a far shorter survey, but added in the option to ask further questions specific to each round, and anyspecific aspects of that round which may have been unique to it. We recorded drop-off rates for each survey to understand how many applicants were giving up part way through. This helped us to gauge whether the length of the questionnaire was related to overall response rate. Individual reports were given to the Scottish Government following each survey conducted so that feedback from applicants for each individual round could be fed back into the Government’s “lessons learned” approach. The “lessons learned” approach stems from the Guidance on the Code of Practice para 4.1 which states:
“The Scottish Government is establishing a mechanism for capturing lessons learned such that a repository of information on good practice in inclusive attraction, application and assessment methods is instituted and added to over time. This will be informed by the experiences of selection panels and the views and demographic data of applicants and appointees.”
Sample size and response rate
The Scottish Government undertook to provide a list of email addresses of applicants for each round who met the following criteria:
- Had applied for a position in the round; and
- Had indicated a willingness to take part in research to inform the appointments process.
A record was kept for each survey sent out of how many applicants it was sent to, how many made an initial response to the survey, and how many completed the survey in full.
This report includes details of all the surveys carried out in 2016. These each took place shortly after the round concluded and therefore the round itself did not necessarily take place during 2016. A list of all the bodies included with respective drop off rates is shown below.
Round / Survey sent to: / Responses at beginning: / Responses at end:Scottish Children’s Reporter Administration(SCRA) members / 53 / 22 / 21
Scottish Legal Aid Board(SLAB) chair / 6 / 4 / 4
Scottish Legal Aid Board(SLAB) members / 72 / 28 / 27
Scottish Legal Complaints Commission(SLCC) members / 91 / 48 / 42
Bòrd na Gàidhlig(BnG) members / 14 / 7 / 7
Office of the Scottish Charity Regulator(OSCR) members / 138 / 65 / 59
Sportscotland chair / 18 / 5 / 5
Grampian NHS members / 45 / 18 / 17
Greater Glasgow and Clyde (GG&C) NHS members / 193 / 88 / 86
Shetland NHS members / 14 / 7 / 7
Western Isles NHS members / 30 / 13 / 13
Highland NHS members / 34 / 14 / 14
Highland and Islands Airports Limited (HIAL) Chair / 7 / 4 / 3
Royal Botanic Gardens Edinburgh (RBGE) Trustees / 6 / 2 / 2
Scottish Water members / 71 / 27 / 26
Water Industry Commission for Scotland (WICS) members / 13 / 4 / 4
Total / 805 / 356 (44%) / 337 (42%)
The Bòrd na Gàidhlig (BnG) survey was supplied in Gaelic and English.
Previous annual surveys saw a response rate of around 35%. As can be seen from the table above, with the move to surveys per round, the response rate has risen to 42% for those who have completedthe whole survey and 44% for those who have completed at least part of it.
Demographic analysis
Of the 337 applicants who completed the full survey, 264 (78%) provided demographic data. In this report, demographic information is presented primarily in relation to the aggregated results. Where appropriate, disaggregated results are shown for women, candidates describing themselves as having a disability, those under 55[1], Black and Minority Ethnic (BME) and Lesbian, Gay and Bisexual (LGB). Results are also shown for first time applicants.
The table below shows the demographic profile of board membership at 31 December 2016 and the percentage of each group responding to the applicant surveys during 2016:
Target Group / Profile of board members† at the end of 2016 / Scottish Population(2011 Census) / Applicant Survey Responses 2016^
Female / 45.1% / 51.5% / 55.7%
Disabled / 9.2% / 19.6% / 20.5%
Black and minority ethnic†† / 3.6% / 4.0%†† / 4.2%
Aged 49 and under (under 55 for survey applicants) / 15.9% / 54.3%* / 41.7%
Lesbian, gay and bisexual / 4.0% / 6.0%** / 6.1%
†All board members inclusive of the chair unless otherwise stated. Percentages do not include those who did not make a declaration.
†† Black and minority ethnic figures reflect people from a non-white minority ethnic background