Project Title:
Assessing the Impact of Labour Market Information (LMI)
Project Number:
9042797
Assessing the Impact of Labour Market Information:
Final Report – Phase III
Results of Analysis of Follow-up Interviews
Research Team:
Réginald Savard, Bryan Hiebert, Céline Renald, Lynne Bezanson, and Sylvain Paquette
Canadian Career Development Foundation (CCDF)
Canadian Research Working Group on Evidence-Based Practice in Career Development (CRWG)
Report Authors
Réginald Savard, Ph.D., Professor
Sylvain Paquette, M.Ed., Research Professional
Collectif de recherche en counseling et développement de carrière
Département d’orientation professionnelle, Université de Sherbrooke
1
FOREWORD
This research report takes stock of the results of a study designed to assess the impact of labour market information on a Francophone population. The study is based on the method and analytical framework developed for a research project that had the same objective but was conducted with an Anglophone population (Hiebert, 2012b).
EXECUTIVE SUMMARY
This study as a whole was designed to answer two research questions: a) To what extent is the independent method a sufficient process for clients to use LMI effectively? and b) To what extent does the assisted method offered by a counsellor enhance the effective use of LMI by clients? Supplementary research questions looked at the following points: a)For clients with a career choice or job search need, what type of assistance in using LMI leads to what kinds of outcomes? and b) What demographic factors (e.g., age, gender, employment history) influence the effectiveness of an LMI intervention? These questions were addressed in a context where client needs were assessed to determine whether LMI for career choice or LMI for job search was the appropriate intervention.
This project isolated the use of LMI, making it an independent intervention unlike training workshops or career counselling. To do this, LMI binders were developed for the two employability dimensions targeted in this study: one binder for career choice and the other for job search. In both cases, the binders included lists of available resources, instructions for accessing information, and tips for interpreting and applying LMI information to the client’s situation.The clients selected for the study were likely to benefit from LMI when choosing a career or searching for a job. Clients who, for one reason or another, seemed to require more than an LMI intervention were not recruited for the study. The LMI delivery method (with or without assistance) was assigned randomly. The approach as a whole, regardless of the LMI delivery method, was designed primarily to promote career self-management, or self-guidance. The program lasted for three weeks.
The results of the analysis of the questionnaires were presented in the report on Phase II of the project. This report focuses on the results of the analysis of the two follow-up interviews conducted with a client subset – the first, one week after the LMI intervention and the second, four weeks after. Generally speaking, the LMI delivered during the intervention was considered relevant by a large majority of clients, who were able to use it in a manner consistent with their initially identified need, i.e., career choice or job search. The clients were optimistic and confident one week after the intervention. Four months later, this remained basically unchanged: clients continued to use LMI, although less frequently; and some said they were confident they could use it again when the need arose during their careers. However, certain criticisms were expressed by the clients who participated in the study. Feeling confused by the abundance of information was mentioned several times, as wasa lack of computer skills, which limited access to LMI resources. Despite those criticisms, the overall results suggest that the LMI intervention promoted the development of career self-management skills.
The results also indicate that a counsellor’s assistance would benefit most clients in two ways: 1)an adjusted understanding of the information relating to self-knowledge and the world of work, and 2)adjusted use of information based on the client’s individual situation. For those reasons, in addition to using the LMI binder, a counsellor’s assistance would be worth considering in connection with LMI interventions, particularly for career choice. Also, a few weeks after the LMI intervention, follow-up should be provided, especially for clients who encounter barriers to employment or whose employment opportunities have not been favourable. Not all clients have the same employability needs. The Council of the European Union (2004, in Michaud, Savard, Paquette Lamarche, 2011) has therefore identified three levels of intensity for guidance services:
- level 1: self-help (little help)
- level 2: initial consultation with a competent counsellor–to clarify needs and identify relevant pathways or facilitate access to pathways (more help)
- level 3:more specialized support, such as career counselling and evaluation (intensive help).
On the basis of these levels and the data collected during the study, it is reasonable to say that the intervention offered as part of this project corresponds to intensity levels 1 and 2.[1]Some of the data collected four months after the LMI intervention suggest that the needs of certain clients were moving towards level 3 after the intervention. Moreover, a certain number of clients took advantage of such assistance.
All studies have methodological limitations, and this one is no exception. First of all, the quantitative measures obtained in Phase II were based mainly on the clients’ perceptions. In addition, the clients’ comments, collected by means of follow-up interviews, were considered on the basis of the corpus of notes taken by the interviewers, and that corpus was made up of all the qualitative data analyzed in Phase III. In addition, the interview protocol did not include any questions about LMI delivery methods (independent or assisted). Last, the composition of the client subsets that responded one and four months after the LMI intervention is not the same, which sometimes limited interpretations according to intervention type (career choice or job search), method (independent or assisted), employment status (unemployed, part-time work, or full-time work), and data collection time (T1 or T2).
Despite these limitations, the analysis of the results of the follow-up interviews makes it possible to validate and sometimes qualify the results obtained using the quantitative measures administered in Phase II of the project. The comments of the clients who participated in theinterviews provide a better understanding of the factors that contribute to a successful LMI intervention.
TABLE OF CONTENTS
FOREWORD
EXECUTIVE SUMMARY
LIST OF TABLES
1.PrEsentation
2.MEthod
2.1.Data Collection
2.2.Data Analysis
2.3.Sample
3.RESULTS
3.1.Results One Week After the LMI Intervention(T1)
3.1.1.Positive impact of LMI
3.1.2.LMI use during the intervention
3.1.3.LMI use after the intervention
3.1.4.Career clarity, optimism, and confidence
3.1.5.Other themes
3.2.ResultsFour Months After the LMI Intervention (T2)
3.2.1.Positive impact of LMI
3.2.2.LMI use after the intervention
3.2.3.Career clarity, optimism, and confidence
3.2.4.Other themes
4.Discussion
5.CONCLUSION
REFERENCES
LIST OF APPENDICES
ANNEXE AProtocole d’entretien de suivi de l’intervention en IMT...... 71
ANNEXE BRésultats relatifs à la méthode autonome (T1)...... 77
ANNEXE CRésultats relatifs à la méthode assistée (T1)...... 104
ANNEXE DRésultats relatifs à la méthode autonome (T2)...... 131
ANNEXE ERésultats relatifs à méthode assistée (T2)...... 147
LIST OF TABLES
Tableau 1. Distribution de l’échantillon au T1 selon le type d’intervention et la méthode...11
Tableau 2.Distribution de l’échantillon au T2 selon le type d’intervention et la méthode...11
Tableau 3.Suivis au T1 et au T2 : statuts d’emploi et satisfaction à l’égard de leur carrière..12
Tableau 4.Attribution pour le statut d’emploi...... 13
Tableau 5.Perceptions des apports positifs de l’IMT (T1)...... 15
Tableau 6.Pensées en cours d’utilisation de l’IMT (T1)...... 20
Tableau 7.Parties d’IMT jugées les plus utiles (T1)...... 22
Tableau 8.Parties d’IMT jugées manquantes (T1)...... 25
Tableau 9.Présence d’un plan d’action et incitatif à son élaboration (T1)...... 27
Tableau 10.Suggestions relatives à l’utilisation de l’IMT (T1)...... 29
Tableau 11.Utilisation de l’IMT à la suite de l’intervention (T1) ...... 32
Tableau 12. Utilisation d’autres ressources d’IMT à la suite de l’intervention (T1)...... 33
Tableau 13.Démarches effectuées à la suite de l’intervention (T1) ...... 35
Tableau 14.Clarté du cheminement de carrière (T1) ...... 38
Tableau 15.Degré d’optimisme et de confiance face à la carrière (T1)...... 41
Tableau 16.Autres thèmes abordés au cours de l’entretien de suivi (T1)...... 44
Tableau 17.Perceptions des apports positifs de l’IMT (T2)...... 48
Tableau 18.Utilisation de l’IMT à la suite de l’intervention (T2) ...... 50
Tableau 19. Utilisation d’autres ressources d’IMT à la suite de l’intervention (T2)...... 51
Tableau 20.Démarches effectuées à la suite de l’intervention (T2) ...... 52
Tableau 21.Clarté du cheminement de carrière (T2) ...... 55
Tableau 22.Degré d’optimisme et de confiance face à la carrière (T2)...... 57
Tableau 23.Autres thèmes abordés au cours de l’entretien de suivi (T2)...... 59
1
1
ASSESSING THE IMPACT OF LABOUR MARKET INFORMATION (LMI): RESULTS OF FOLLOW-UP INTERVIEWS
1.Presentation
The results of the follow-up interviews make up Phase III of the research project assessing the impact of labour market information (LMI)on career choice and job search. This phase consistedprimarily of a detailed exploration of the nature of the clients’ experiencesin using LMI carried out through semi-directed telephone interviews with a subset of clients who used either the independent or the assisted LMI delivery method. The interviews focused on how clients accessed and used LMI, with attention to the details of their experiences in relation to how they processed the LMI they accessed. This report presents the results of Phase III of the project. In Phase I, a snapshot of the state of LMI practices at different employment centres across the country was produced. Phase II compared two methods for delivering LMI to clients:an independent method whereby clients accessed LMI on their own and an assisted method whereby clients again accessed LMI on their own but benefitted from the assistance of a counsellor.Both LMI delivery methods – independent and assisted – were used with adult clients with career choice or job search needs.
Overall, as was already pointed out in the earlier reports, the project used a practitioner-researcher approach whereby the research is incorporated into the usual intervention practices of organizations that help with employability development. A number of authors were critical of the fact that the research results were not sufficiently incorporated into the daily practice of the counsellors, likely because those counsellors did not find the experimental methodologyto be useful or applicable (see Feller, 1993; Sexton, 1993; Walz Bleuer, 1993). One of the measures that might solve this problem is that of adopting an experimental methodology that corresponds closely to what is normally done in the field. Accordingly, this research was implemented in centres where actual services are offered. It incorporated best practices for using LMI in those contexts, it had professionals at those centres offer the interventions, and for the research participants, it recruited clients who had asked for help with their career choice or job search needs. It is reasonable to believe that this approach will increase the probability that any positive results will be incorporated into regular practice.
The results of Phase I and Phase II were presented earlier in the reports on those phases (Hiebert, Bezanson, O’Reilly, Hopkins, Magnusson McCaffrey, 2010; Hiebert, 2012a; Savard Paquette, 2012). This report focuses on the results of PhaseIII. It is worth issuing a reminder of some of the terms that are used in this document to ensure compliance with the Phase II report. The term used for the intervention offered to clients,LMI intervention, indicates that the intervention focused on the use of LMI (with or without assistance). The term intervention type refers to the clients’ needs, i.e. career choice or job search. It should be noted in passing that those needs correspond to employability dimensions that counsellors assess at the start of the intervention. Lastly, frequent mention will be made of independent groups or clients and assistedgroups or clients. These terms refer to the two methods of LMI delivery during the intervention. Clients in the independent group accessed LMI on their own, whileclients in the assisted group accessed LMI on their own but also received a counsellor’s assistance.
2.MEthoD
2.1Data Collection
Client perceptions were obtained through semi-structured telephone interviews. Data was collected twice: T1)one week after the intervention, and T2)four months after the intervention. The goal of the follow-up interviews was to gain a better understanding of the factors the clients might have dealt with in their responses to the questionnaires administered in the previous phase of the project. The follow-up interviews focused on the clients’ experiences in accessing and using LMI;how they processed, personalized, and interpreted the information accessed; what resources were considered the most useful; what information might have been useful but was not provided; and how well prepared they thought they were to manage their own future career transitions. The same protocol was used for both interviews, although slight adjustments were made to the T2 interviews to reflect the time that had elapsed since the end of the intervention. The interview protocol is found in Appendix A.
Most of the T1 interviews were conducted by two research assistants. The T2 interviews were done by a research assistant who had conducted many of the T1 interviews. During the interviews, the interviewers were careful to take notes documenting the clients’ responses to the various questions. Those notes were the principal source of data for analysis. In order to reduce the degree to which the interviewers’ perceptions might influence the recording of the clients’ comments, an initial meeting was held to present how the interviews were to be conducted and the notes taken. The interviewers stayed in touch with the project coordinator, who ensured note-taking uniformity. Lastly, since only one experienced assistant conducted the T2 interviews, it is reasonable to think that the clients’ responses were reported in a consistent manner.
2.2Data Analysis
Once the data were collected, the content was analyzed to identify recurring themes in the clients’ responses. The interview notes were coded in digital form and collated in a database using QSR Nvivo9 software. A tree structure, orhierarchy of categories based on the topics discussed during the interviews, was developed using the topics and categories that were identified in the Anglophone portion of this project (see Hiebert, 2012b). This will make it possible to compare the two population groups. Unless otherwise indicated, the subcategories for a given topic are mutually exclusive. Most of the answers to a question are coded only to the corresponding nodes, except for certain themes that could have been discussed at any time during the interview. A series of documents was coded by a research professional, who coordinated thecollection of the data. Four of those were selected and coded by a research assistant. The two persons who did the coding then got together to discuss and standardize it, and if necessary, to change the categories. The coding was completed by the research assistant, who consulted the first coder, if need be.
2.3Sample
The sample of clients who participated in the first follow-up, i.e., one week after the intervention (T1),consisted of 39 persons: 19 men (49%) and 20 women (51%), giving a gender distribution comparable to that of the overall research sample. Within that sample, 25 persons (64%) took part in the LMI intervention for career choice and 14(36%),in the one for job search. With regard to LMI delivery method, 21persons (54%) used the independent method and 18 (46%), the assisted method, for a distribution that is also comparable to that of the overall sample.
Tableau 1
Distribution de l’échantillon au T1 selon le type d’intervention et la méthode
MéthodeAutonome / Assistée / Total
Type d’intervention / Choix de carrière / 13 / 12 / 25
Recherche d’emploi / 8 / 6 / 14
Total / 21 / 18 / 39
A total of 27 clients participated in the second follow-up (T2), four months after the intervention: 14 men (52%) and 13 women (48%), with 20 (74%) taking part in an LMI intervention for career choice and 7 (26%) in one for job search.With regard to LMI delivery method, 17 persons (63%) used the independent method and 10 (37%), the assisted method. Of those who participated in the second follow-up, two had been unable to participate in the first one.
Tableau 2
Distribution de l’échantillon au T2 selon le type d’intervention et la méthode
MéthodeAutonome / Assistée / Total
Type d’intervention / Choix de carrière / 12 / 8 / 20
Recherche d’emploi / 5 / 2 / 7
Total / 17 / 10 / 27
During each follow-up (T1 and T2), the clients were asked to describe their current employment situation and, if they were working full or part time, they were also asked to indicate how well their job fit with their career vision. Table 3 presents those data. Just under half of the subset of clients who participated in the second follow-up (44%) were working full time, and most said their job fit with their career vision. Another 11% were working part time, but most of them said their job was not a good fit with their career vision.
Tableau 3
Suivis au T1 et au T2: statuts d’emploi et satisfaction à l’égard de leur carrière
Suivi T1 / Suivi T2Cadre bien / Cadre à peu près / Cadre mal / Statut d’emploi / Cadre bien / Cadre à peu près / Cadre mal / Changement de statut d’emploi
N/A / N/A / N/A / Sans emploi / N/A / N/A / N/A / SeSe=10
TpaSe=0
TplSe=11
25 (64%) / (Se) / 12 (44%) / 111
0 / 4 / 1 / Travail à temps partiel / 1 / 0 / 2 / TpaTpa=2
TplTpa=1
SeTpa=0
5 (13%) / (Tpa) / 3 (11%) / 3
2 / 3 / 4 / Travail à temps plein / 7 / 3 / 2 / TplTpl=4
TpaTpl=1
SeTpl= 6
9 (23%) / (Tpl) / 12 (44%) / 111
39 / Total / 27
Notes. Le libellé de la question est: «Dans quelle mesure votre emploi cadre-t-il avec l’idée que vous vous faites de votre carrière ?» N/A=non applicable. Les pourcentages totaux peuvent ne pas atteindre 100% en raison de l’arrondissement. 1Une personne a répondu au T2, mais pas au T1.
Table 3 also contains data on changes in employment status between the first and second follow-ups (e.g., between full-time status one week after participating in the program and part-time status four months after).Theemployment status abbreviations used in the various tables in this report are as follows: 1)Ue=unemployed, 2)Ptw=part-time work, and 3)Ftw=full-time work. The original employment situation of most of the 25 clients who participated in both follow-up interviews had not changed: 4were still working full time, 2were working part time, and 10 were still unemployed. Without taking job fit into account, the employment situation of several clientshad improved: four months later, 6 unemployed clients had found a job, and 1 had gone from part-time work to full-time work. Still without taking job fit into consideration, the employment situation of a minority had apparently worsened: 1 client had gone from full-time work to part-time work, and another was no longer employed after having had a full-time job one week after the intervention.