EvolutionResearch
Department of Social Services
Career Development Tool (CDT) Project
June 2014
Final
Evolution Research
PO Box 558 Glenelg SA 5045
Phone 0411 586538
Email:
June 2014
Executive Summary
Background
This project was commissioned by the Department of Social Services (DSS) to train and assist a small number of Disability Employment Service (DES) Providers who volunteer to trial a career assessment tool and to evaluate the process through feedback from DES consultants and participants who are involved in the project.
A primary objective of DES is to achieve sustainable employment outcomes for participants. Employment Pathway Plans (EPP) and effective job matching are considered central to this aim. The evaluation of DES 2010-2013 found that:
- There is low satisfaction with training and skills development, with many participants not getting the development opportunities they believed they needed;
- There are very low education outcomes, with the focus almost entirely on employment outcomes despite a relatively low skills base;
- Good job matching is of importance to achieving sustainable outcomes, especially for people with high support needs; and
- Many participants who were placed in employment would prefer a job with better prospects.
These findings presented a strong rationale for exploring ways to target skills development and develop job matching capability in a service delivery setting. DSS therefore commissioned Evolution Research to conduct action research as a first step to examining the potential benefits of career assessment instruments for employment pathway planning for people in DES.
Project description
The requirement of the project involved training and assisting a small number of DES providers who volunteered to trial career assessment instrument/s and to evaluate the process through feedback from research participants. The project required participating providers not currently using such a tool to be equipped with a career development resource and to then assess whether this assists in identifying the skills and capabilities of job seekers to better inform skills development needs in the EPP. In this way the project design was intended to simultaneously deliver a tangible benefit whilst also researching the role of career development tools in employment service planning.
Training was provided to 19 consultants from four DES providers based in metropolitan New South Wales (Sydney). The number of training participants was limited only by DES consultant availability as DES providers were offered the option to nominate as many staff as possible. Training sessions were held at each DES provider’s office. Material covered in the training included:
- Introduction to the project and use of standardised assessment tools for career development;
- Administration and interpretation of the chosen career development tool; and
- Labour market research and identification of realistic career goals.
On completion of training, DES consultants had a period of approximately four weeks in which to trial the career development tool with as many DES participants as possible, as part of their employment pathway planning process.
Following the trial period, DES consultants were required to attend a focus group to provide feedback regarding their perceptions of the usefulness of career development tools. At this time consultants also provided Evolution Research with de-identified client profile information which was used to provide background understanding of the rationale to choosing which clients may benefit from the use of a career development tool, and also background information regarding participant disability types, education and employment history, duration with the DES and duration unemployed. DES consultants also provided de-identified copies of participant EPPs for identification of how the career development tool had impacted on EPP development. Individual face-to-face interviews were also held with DES participants who agreed to meet with an Evolution Research consultant to gain feedback regarding their satisfaction with the employment pathway planning process incorporating a career development tool. Feedback gained from DES consultants and participants was also used to identify any issues that were encountered in using a career development tool in a DES setting.
Findings
The use of a career development tool as part of the employment pathway planning process has been very well received by both DES consultants and DES participants. DES consultants stated overwhelmingly that the use of career development tools had added value to the employment pathway planning process and that it was found to be of great assistance in identifying realistic employment and study options with participants. Similarly, the majority of DES participants who participated in interview indicated the same findings.
Both DES consultants and participants felt that inclusion of a career development tool in the employment pathway planning process is beneficial as it assisted in identifying alternate job options that may be suitable and also assisted in identifying actions that would need to be taken in order to reach an identified and desired employment or study goal.
However, in the majority of cases participant EPPs had not been updated to reflect or include specific details of the actions that they had identified with the DES consultant in order to achieve their employment or study goals. The majority reported that the EPP is primarily used to provide a broad overview of the actions required when participating in DES and that it does not provide specific detail as to the actual activities that would be undertaken.
Some barriers were identified in relation to the use of a career development tool in a DES setting, for example when working with participants who present with characteristics such as extremely low motivation and interest in activities, extremely low insight and confidence in their skills or extreme ambivalence in relation to returning to work. These barriers are not considered specific to the tool used as part of this trial, but are considered to highlight issues that require implementation of alternate interventions such as counselling or motivational interviewing, in order to assist the participant to become ready to consider employment options and to participate in completion of a career development tool.
Other barriers that were identified included that participants of an older age group (e.g. age in their mid-fifties to sixties) and also participants with a history of high level qualifications and highly skilled employment history, gained less benefit from the tool. These participants indicated that the tool did not effectively target higher-skilled jobs and felt that an alternate approach, such as individual career counselling, may be more effective.
Overall, both DES consultants and participants indicated that a career development tool is suitable to use with most participants in a DES setting and that it is a valid and useful approach to assisting participants to determine realistic employment and study goals.
Despite the limited timeframes and competing demands which prevented broader application by DES consultants with participants over the project period, it is considered that the findings of the project provide a useful indication that the use of a career development tool as part of an employment pathway planning process in DES settings warrants further investigation. The use of a tool provides a tangible and evidence-based point from which to commence discussions with participants who are uncertain or unrealistic about their future job choices and once potential jobs are identified, a solid basis from which to identify skills development needs.
There is clear potential from the findings that the usefulness of the EPP to DES participants can be improved significantly, by making it a working planning document that directly links the goals and specific activities that participants need to undertake to work towards their study and/or employment goals. Further training to DES consultants for this purpose would be considered beneficial.
On the basis of the early findings of this project, it is considered likely that if the use of a career development tool could be embedded into DES practices, ensuring that DES consultants are appropriately trained not only in a tool’s use but also in appropriate job matching, skills development exploration and more effective utilisation of the EPP, that many of the adverse findings of the DES 2010-2013 evaluation could be overcome.
Contents
Executive Summary
Background
Project description
Findings
Introduction
Background to the project
Purpose of the project
Methodology
Selection of Career Development Tool
OSI-2
CDT Training Strategy
Training participants
CDT Training
Project Requirements
Identification of DES participants
Data Collection
Findings
Participant Profile
Process of profiling participants
DES Consultant Feedback
Usefulness of a career development tool
Impact of career development tools on future employment pathway planning
Participant Feedback
Experience and satisfaction using the career development tool process
Usefulness of a career development tool for employment planning
Identification of next steps in the employment pathway planning process
Appropriate application of a career development tool
Issues or barriers experienced in using a career development tool in a DES setting
Application in practice
Discussion
Overall Findings
Perceptions of DES consultants of the usefulness of career development tools
DES participant satisfaction with the use of a career development tool
Issues experienced in using career development tools in DES settings
Limitations of the current project
Conclusions
Recommendations
Appendices
Focus group discussion guide
Client interview guide
Acronyms
ADHD – Attention-Deficit Hyperactivity Disorder
ASCO – Australian Standard Classification of Occupations (ASCO)
CDT – Career Development Tool
CISS – Campbell Interest and Skills Survey
CSS – Congruence Skills Sort
DES – Disability Employment Service
DSS – Department of Social Services
EPP – Employment Pathway Plan
HA – Harrison Assessments
JVIS – Jackson Vocational Interest Survey
KPI – Key Performance Indicator
MBTI – Myers-Briggs Type Indicator
NEO PI-R – NEO Personality Inventory
OCD – Obsessive Compulsive Disorder
OSI-2 – Occupational Search Inventory – Form 2
SDS – Self-Directed Search
SII – Strong Interest Survey
Introduction
Background to the project
A primary objective of Disability Employment Services (DES) is to achieve sustainable employment outcomes for participants. Employment Pathway Plans and effective job matching are considered central to this aim. The evaluation of DES 2010-2013 found that:
- There is low satisfaction with training and skills development, with many participants not getting the development opportunities they believed they needed;
- There are very low education outcomes, with the focus almost entirely on employment outcomes despite a relatively low skills base;
- Good job matching is of importance to achieving sustainable outcomes, especially for people with high support needs; and
- Many participants who were placed in employment would prefer a job with better prospects.
These findings presented a strong rationale for exploring ways to target skills development and develop job matching capability in a service delivery setting. The Department of Social Services (DSS) therefore commissioned Evolution Research to conduct action research as a first step to examining the potential benefits of career assessment instruments for employment pathway planning for people in DES.
Purpose of the project
The requirement of the project involved training and assisting a small number of DES providers who volunteered to trial career assessment instrument/s and to evaluate the process through feedback from research participants. The project required participating providers who are not currently using such a tool to be equipped with a career development resource and to then assess whether this assists in identifying the skills and capabilities of job seekers to better inform skill development needs in the Employment Pathway Plan (EPP).
Theintention of the EPP is tounderpin the provision of services to a job seeker[1]. The EPP is an individually tailored plan that identifies the job seeker's vocational goals, and records the activities the job seeker will undertake to reach these goals. If a job seeker has an employment services provider the plan will also list specific ways the provider will support a job seeker to improve their employment prospects and gain sustainable employment.
The EPP should be reviewed and updated at each contact with an employment services provider to reflect changes in the individual's circumstances, participation requirements and the activities they are undertaking.
The EPP caters for all job seekers with activity test or participation requirements and also non-activity tested job seekers. Government policy requires all recipients ofDES(job seeker), who have activity test or participation requirements and do not have an activity test exemption, to have a current EPP. Under social security law, income support recipients with participation or activity test requirements cannot qualify for payment unless they are prepared to enter into and comply with the terms of an EPP when required.
The project design was intended to simultaneously deliver a tangible benefit whilst also researching the role of career development tools in employment service planning.
The overall requirements of the project required Evolution Research to report on:
- Perceptions of DES employment consultants regarding the usefulness of career development tools;
- DES participant satisfaction with the employment pathway planning process incorporating a career development tool; and
- Any issues experienced in using career development tools in DES service settings.
To deliver the project, Evolution Research was required to:
- Identify an appropriate career development tool for the purpose of the project;
- Develop and deliver training to DES provider staff (employment consultants) in the administration of a career development tool;
- Collect feedback from DES consultants on the usefulness of systematic career development planning as part of developing employment pathway plans, using focus groups and structured written feedback; and
- Collect feedback from DES participants on satisfaction with the employment pathway planning process which incorporates a career development tool, using face-to-face interview and written survey.
The research was not an evaluation of the efficacy of the specific career development tool used, but an indication of whether a career development tool in general may be useful in a career development planning process.
Methodology
Selection of Career Development Tool
A number of tools were researched and considered for use in the current project. These included the Occupational Search Inventory (OSI-2 and -3), Self-Directed Search (SDS), NEO Personality Inventory (NEO PI-R), Myers-Briggs Type Indicator (MBTI), Congruence Skills Sort (CSS), Jackson Vocational Interest Survey (JVIS), Strong Interest Survey (SII), Harrison Assessment (HA) and Campbell Interest and Skills Survey (CISS). A number of these tools were determined to be unsuitable for the project’s purpose due to requirements unsuitable for the disability employment setting and range of qualification levels of DES consultants. Several tools were considered unsuitable due to the requirement for the test administrator to hold formal qualifications (e.g. psychologist or relevant university qualification) or specific certification in the tool in order to be eligible for its administration (e.g. MBTI, JVIS, NEO PI-R, SII). Others were considered less suited to the disability employment setting due to limitations in the target application group, for example the CISS which is considered to be primarily useful for career planning for college-bound or college-educated individuals. Tools such as the MBTI and NEO PI-R were also considered unsuitable as the additional experience required to interpret the findings of the tool in order to achieve meaningful career recommendations is considered an unrealistic expectation of the average DES consultant.
Tools which were considered suitable for application in the CDT project included the OSI-2, CSS, SDS and HA tools. The final choice of tool was based on several factors including appropriateness to the DES participant target group, ability to generate a list of potentially suitable jobs from which participants and their consultant could research and choose from, job listings and questionnaire based on Australian norms and Australian job requirements, flexible administration method (paper-based and online), and cost. Two equally suitable tools were identified as the OSI-2 and the SDS, however the OSI-2 was chosen due to it being a more cost effective option in the longer-term.
OSI-2
In completing the OSI-2 questionnaire, test takers rate four work related categories: work activity interests, work skills, job preferences and personal abilities. Eight broad interest categories are assessed across these four work related categories, including: Technological, Nature, Scientific, Culture, Entertainment, Helping, Persuasive and Organising.
Obtained test scores are used to generate an OSI Code which is then related to occupations with similar codes. In this way a variety of occupational options can be listed for the test taker to consider. The OSI Occupations Finder has a list of corresponding occupations for each code (and each permutation of the code) which the test taker can consider for suitability and further investigation. The occupational content of the OSI Occupations Finder is drawn from the Australian Standard Classification of Occupations (ASCO), various other occupational databases and a wide sampling of Australian newspaper job advertisements.
Although the OSI does not require specific qualifications for administration, the test publisher seeks assurance that the purchaser has some training in the principles of testing and some experience in using and interpreting standardised test material. Relevant information in the principles of testing and in the using and interpreting of standardised test material was incorporated into the CDT training provided to DES consultants. It was also noted that any remaining concerns regarding an individual’s suitability to administer the test could be largely overcome by utilising the online administration and scoring system, thereby removing any chance of administrator error.