Implementation Review Findings

Data Exchange
Client Survey Pilot

Implementation Review Findings

31 August 2017

Page 1 of 3

1.  Background

The Department of Social Services is committed to improving the lifetime wellbeing of people and families in Australia. Outcomes data contributes to this commitment by providing an evidence base for understanding how funded services are meeting the needs of individuals and communities.

The client survey forms part of the Data Exchange priority requirements. It was developed to provide data that will allow organisations to report on the following outcomes related performance indicators:[1]

·  KPI 4 – Percentage of clients achieving individual goals related to independence, participation and wellbeing; and

·  KPI 5 – Percentage of clients achieving improved independence, participation and wellbeing.

Organisations which have opted into the partnership approach are also expected to offer the survey to their clients, to capture supplementary information in the form of an independent ‘client voice’.

The client survey pilot was launched in mid-October 2016 with the public release of the Client Survey Discussion Paper. The purpose of the pilot was to evaluate the client survey methodology. All organisations which deliver services in scope for Data Exchange reporting were invited to participate in the pilot. By May 2017, 63 organisations had joined the pilot.

The Department committed to review the implementation of the pilot, and the findings captured in this report are based on:

·  30 submissions in response to the Client Survey Discussion Paper;

·  Feedback provided at training workshops (October-December 2016);

·  Liaison and consultations with organisations (October 2016-March 2017);

·  Feedback provided during Data Exchange National Roadshow workshops (May-June 2017); and

·  Survey metadata as at 30 June 2017.

2.  Findings

As at 30 June 2017:

·  32 out of 63 organisations had commenced survey delivery;

·  537 clients had agreed to complete a survey (510 pre survey and 27 post survey); and

·  463 clients had submitted their survey responses (440 pre surveys and 23 post surveys).

The lower than anticipated number of survey completions signals that further investigation into the question set, design and methodology is required before the survey is fully implemented. While the sample of completed surveys is lower than desired, early trends indicate that a high rate of clients complete the full survey if they do agree to participate and the pre and post survey does measure changes in client outcomes over time.

Qualitative feedback suggests that organisations value the opportunity to capture client outcomes and client satisfaction results. Stakeholders also believe that providing an independent client voice is valuable and strengthens the accountability of organisations. The key areas for improvement identified through consultations and client participation patterns, and the Department’s response, are set out under each of the review criteria:

1.  The survey is accessible to clients

The Department has committed to a plain English review and consideration of the questions’ cultural appropriateness as well as to investigate other survey modes that will strengthen its accessibility for clients.

2.  The survey is able to be integrated into business practice

The survey delivery method will be simplified by introducing a one-off survey and by piloting additional survey modes.

3.  The survey data is fit for purpose

The Department has committed to further investigations into survey tools and material to support implementation of the survey in a way that suits organisations’ business practices.

3.  Conclusion

The Department received feedback during stakeholder consultations during October 2016 to May 2017 which signalled that changes to the current survey questions and survey delivery mechanisms would be welcomed by organisations.

The feedback was consistent with the Data Exchange National Roadshow workshops held in May-June 2017. At the workshops, the Data Exchange training team sought pilot organisations feedback on proposed options for future client survey development. A total of 472 participants indicated their preferences for 10 proposed client survey developments. Theoption of a one-off survey was the most popular, and received 16 per cent of ‘votes’. Workshop participants were equally divided in their preferences for paper, offline, audio, email, translated and telephone assisted surveys (each of these accessibility features received between 10-11 per cent of ‘votes’). This feedback was reinforced by metadata trends across the pre and post survey.

The Department has committed to extend the pilot until early 2018 to allow for the one-off survey to be included in the pilot and investigation into a range of accessibility options.

Organisations are encouraged to sign up for client survey training and join the pilot as it enters this new phase of development.

Client survey enquiries

All enquiries about the Data Exchange Client Survey, including interest in joining the pilot should be directed to .

Page 1 of 3

[1] While the measurement constructs remain the same, the wording of these KPIs in grant agreements varies slightly to clearly align with the objectives of the program activity being delivered. Organisations seeking to identify the exact wording of their KPIs should consult their grant agreement.