HSE Data Workshops 2008 – Summary of issues raised

Introduction

This document has been prepared to provide a summary of the issues discussed at a series of Data Workshops held at HSE area offices in the spring of 2008. The source material has been drawn form flip charts, evaluation forms plus reflections from the presenters.

The following sections reflect the order in which material was presented and summarise the ensuing discussions.

Session 1: The HSE Analysis Tool

The aim of this session was to demonstrate the functionality and usability of the Analysis Tool version 151.

There are two common errors that users make when first using the Analysis Tool:

(i) They fail to set the correct macro security level before opening the tool. The macro security level needs to be set to ‘medium or low’. To do this within Excel you need to select “tools-macro-security” from the top Excel drop down menu.

(ii) Some users have tried to run the Analysis Tool from the HSE Website. You need to download the analysis tool from the website and save a copy to your own PC or network before using the tool.

A demonstration of the Analysis Tool functionality covered the following:

  • Manual configuration of categories
  • Import of categories from file
  • Import data from previous versions of the analysis tool
  • How to cut the data from the ‘raw data’ worksheet and how these data are displayed on the various worksheets; summary of results, question by question, totals.
  • A discussion about benchmarking and the 2004 survey of individuals (PWC2004) and the soon to be released Organisational Data.

There was a large variation in delegate experience/knowledge in respect to the use of common Excel functions. It was noted that to use the Analysis Tool only a rudimentary knowledge of Excel functions is required.

Session 2: Using your data effectively

Thefirst part of this session looked at what data sources are available to help identify areas of concernin organisations. The discussion focused on firstly identifying the various data sources and then discussed the possible challenges associated with the use of these data.

The list of data identified was fairly constant across each of the workshops and included the following:

  • Surveys
  • Exit questionnaires/interviews
  • Occupational health data
  • Sickness absence data
  • Accident statistics
  • Performance reviews
  • Return to work interviews
  • Organisational performance data
  • Grievance and disciplinary figures
  • Recruitment and retention data
  • Time/flexi sheets
  • Complaints

Discussion on the relative use of such data indicates a general feeling that much of it is of poor quality. That is, the data is not collected in a consistent manner and therefore may not be representative or reliable. It was felt that although these data exist in many large organisations they are often difficult to use.

It is for this reason that the general rule in using such data is not to use just one source of data but to use multiple sources (with a suggested minimum of three), and look for correlations within these. Taking this course of action will increase the probability of arriving at a more accurate diagnosis of the issues.

The second part of this session looked at the ‘Use of demographic data’.

Demographic data(or ‘categories’ as they are referred to in the HSE Analysis Tool) are important. If selected appropriately, demographic data can increase the usefulness of the various data sources listed above.

Some of the demographic categories identified through discussion at the workshops are as follows:

  • Full time/part time staff
  • Permanent / contract
  • Home / office based
  • Length of service
  • Job function
  • Department
  • Working patterns
  • Age
  • Gender

The collection of demographic data relative to a particular workplace can help make better use of the data sources discussed in the first part of the session. For example, results from a staff survey can be cut to identify a specific group of workers and then compared with e.g. sickness absence data, turnover and/or performance data for the same group. It would appear that there is a general lack of awareness/knowledge in this type of data analysis.

Session 3: Understanding your data

Part One - Interpreting Your Data.

This discussion was focused on how to interpret the results from the HSE Analysis Tool, by using a set of fictitious results. The aim was to demonstrate how demographic categories can be used within the analysis tool to cut the data in different ways, tohelp identify issues for different groups within the organisation. The exercise demonstrated how easy it is to cut data using the Analysis Tool providing, of course, that you gathered appropriate demographic category data!

The main message from this session was that if you are planning a new survey using the HSE Indictor Tool or other survey instrument, it is important to give careful consideration to what demographic categories to use. Thinking about the structure of your organisation and how you intend to use the results of the survey can help inform your choices. In addition, also consider what demographic categories have been used to collect other data in your organisation (sickness absence, turnover etc.) as this will help when comparing different data sets.

Part Two - Assessing Intervention Success

This discussion clarified the HSE position on the use of the Indicator / Analysis Tools as a means of assessing the impact of interventions. As with earlier discussions on the use of data, the general message is to not rely solely on a single source of data. Ideally the design of an intervention should include consideration of how its impact will be measured. Normally this can be achieved by collecting qualitative and/or quantitative data. However, if the intervention was aimed at a specific set of workers then you may want to simply ask them if things have improved. While this sounds obvious it would appear that many organisations rely heavily on quantitative data rather than engaging directly with their employees. The risk ofrelying solely on quantitative data is that the measure you are using may not be sensitive enough to pick up small improvements.

Session 4: Improving response rates

This short session focused on factors that could affect the response rate to staff surveys. The issue with a low response rate is that the results will not represent the views of the whole workforce and therefore will be of limited use. In general terms, response rates to staff surveys can vary widely - We are aware of variations in response rates from 10% of staff to 95%! The key to improving response rates seems to be effective communications. This includes ensuring employees understand the purpose of the survey, what the results will be used for, how and when they will receive feedback etc. Timing is also important - both the timing of the communications above and the timing of the survey itself. Consideration should also be given to avoiding holiday periods and other sector specific parts of the year where staff are unlikely to be able to complete a survey.