HighBeam Research

Title: Evaluating training programs: an exploratory study of transfer of learning onto the job at Hotel A and Hotel B, Sydney, Australia.
Date: 4/1/2004; Publication: Journal of Hospitality and Tourism Management; Author: Putra, Andreas

Holton III, Bates, Seyler and Carvalho (1997) define transfer of learning as the degree to which trainees apply to their jobs the knowledge, skills, behaviours and attitudes they gained in training. One of the focal points of the evaluation of transfer of learning is the investigation of the factors which influence how transfer takes place. Trainees in two Sydney hotels were interviewed to find out what methods and strategies they used to enhance their own transfer of learning. This study applied a case study approach and the results of the interviews indicate that there is no one single best strategy, but that different trainees use different strategies that are suitable to their work environments to aid their transfer of learning. One major finding was that all the trainees interviewed had knowledge of the importance of transfer of learning to the job.

**********

Evaluating the effectiveness of training programs is essential for managers who need to ensure that trainees apply the learning gained from training to their work situations (Kirkpatrick, 1998). This has been an imperative for a considerable time (Huczynski & Lewis, 1980). Training effectiveness can in part be determined by examining the transfer of learning to the work environment (Holton III, Bates, Seyler, & Carvalho, 1997; Kirkpatrick, 1998; Noe, 1986). Since Kirkpatrick's evaluation levels (reaction, learning, behaviour and results) were first introduced in 1959, evaluations of transfer of learning have included examining the characteristics of training programs and individual trainees, and then relating these factors to training and job performance (Rouiller & Goldstein, 1993). Kirkpatrick (1998) has found that many managers have only evaluated training programs at the first level (reaction) by using "smile sheets", because it is quick and simple. However, this is inadequate (Holton III, 1996). Instead, job performance also needs to be considered and this is related to training program outcomes through transfer of learning. Training evaluation should involve applying all four of Kirkpatrick's evaluation levels and be focused on whether the learning is transferred by the trainee who, as a result, becomes more efficient and effective in their work performance and, in consequence, improves the performance of the organisation as a whole (Huczynski & Lewis, 1980; Kirkpatrick, 1998).

This article focuses on the second of Kirkpatrick's levels of evaluation, specifically the transfer of learning. The aim of this article was to investigate how trainees went about transferring their training program learning to their work situations.

Literature Overview

A critical issue with any training program is the successful transfer of learning on to the job (Ford, Quinones, Sego, & Sorra, 1992; Kirkpatrick, 1998). The study of transfer of learning is important to all organisations, which are concerned not only with increasing productivity and performance improvement but also with the way people interact, that is, how they relate to one another (Mbawo, 1995). In the context of an organisation's training program, the word "transfer" refers to a trainee's application to the job of what is learned (Burke, 1997). Transfer of learning, in this context, can be defined as the degree to which trainees apply to their jobs the knowledge, skills, behaviours and attitudes they obtained in training (Baldwin & Ford, 1988; Burke & Baldwin, 1999; Holton III et al., 1997; Kirkpatrick, 1998; Tannenbaum & Yukl, 1992).

The criterion of successful training is the application of the newly acquired skills and knowledge to the workplace (Holton III et al., 1997). Baldwin and Ford (1988, as cited in Holton III et al., 1997), Goldstein (1986) and Wexley and Latham (1991, as cited in Holton III et al., 1997) have highlighted a number of factors that might impact on transfer of learning, and one of these is the extent to which the trainee is given the opportunity to perform the tasks taught in training sessions on the job. According to Holton III et al. (1997) and Ford et al. (1992), the opportunity to perform depends upon the work experiences of trainees after the training, the tasks set by the supervisor and the effort of trainees to obtain work experiences relevant to the tasks for which they were trained.

When conducting an evaluation of a training program, managers need to consider several factors which occur before, during and after training and which influence transfer of learning (Baldwin & Ford, 1988).

* Before training: individual characteristics and trainees' motivation to attend may effect learning transfer (Ford et al., 1992)

* During training: discussions to encourage trainees' enthusiasm/willingness to apply new skills may affect transfer of learning (Huczynski & Lewis, 1980)

* After training: the work environment can encourage (e.g., rewards, job aids), discourage (e.g., ridicule from peers), or actually prohibit the application of new skills and knowledge on the job (Ford et al., 1992; Huczynski & Lewis, 1980; Tannenbaum & Yukl, 1992, as cited in Tracey, Tannenbaum, & Kavanagh, 1995).

According to Tracey et al. (1995), the period after training is considered the crucial time to facilitate positive transfer. There is evidence that also argues for the importance of the work environment in effecting transfer of learning. For example, Baldwin and Magjuka (1991, as cited in Tracey et al., 1995) demonstrated that there were three organisational factors that positively influence trainees' intentions to apply what they have learned to their jobs. These are (1) when trainees received relevant information before the training program, (2) when trainees recognised they would be held accountable for learning and (3) when trainees perceived training as mandatory. The time period directly after training appears most suitable to evaluating the outcomes of transfer of learning (Wexley & Baldwin, 1986). Several authors suggest that post-training transfer interventions must be explored to ensure that training programs have an impact on job performance (Baldwin & Ford, 1988; Noe & Ford, 1992; Tannenbaum, Mathieu, Salas, & Cannon-Bowers, 1991; Tannenbaum & Yukl, 1992).

Although some conceptual work has been done in this area in recent years, rigorous empirical investigation of transfer of learning remains scarce (Burke, 1997). Several studies that have attempted to evaluate transfer of learning emphasised numerical findings (Ford et al., 1992; Holton III et al., 1997; Kirkpatrick, 1998; Rouiller & Goldstein, 1993). However, this article argues that statistical measures alone cannot portray the quality and nature of the outcomes of transfer of learning. Cooke (2000) pointed out that we deal with people, so getting participants' comments about training programs will better help to determine the effectiveness of training programs.

However, exploring stories has never been a popular method of training evaluation research and collecting trainees' points of view has been given little attention. Even though a recent review of training research revealed that most investigations of training success have measured the amount of learning that has occurred by the end of a training session rather than on the job performance, little research has been done on exploring the stories of trainees once they return to their workplaces (Baldwin & Ford, 1988, as cited in Holton III et al., 1997; Cooke, 2000; Kirkpatrick, 1998). Asking trainees about training programs is imperative to ensure that trainees have learned something and to help to ascertain whether the training program was beneficial (Cooke, 2000; Kirkpatrick, 1998). Therefore, in order to provide reliable information for the debate on how transfer of learning can be maximised, this article delves into the stories of trainees, which demonstrate how training activities have helped them perform their jobs.

Research Methodology

This exploratory study investigated trainees' experiences of transfer of learning to their workplaces at Hotel A and Hotel B.

This article used a qualitative case study approach to the research process. The major reason for using the case study approach was to focus on specific issues and actions rather than on abstract theories (Mbawo, 1995). According to Wood and Catanzaro (1988, p. 533) case study can be defined as:

an intensive, systematic investigation of a single

individual, group, community, or some other

unit, typically conducted under naturalistic conditions,

in which the investigator examines in

depth data related to background, current status,

environmental characteristics and interactions.

As Stake (1995) stated, a case study may be conducted when the aims are understanding, extension of experience and increase in conviction in that which is known. There is a dearth of recorded research findings investigating trainees' stories in the hospitality industry. Furthermore, Yin (1989, p. 14) suggested that "the case study allows an investigation to retain the holistic and meaningful characteristics of real life events". This is a reason for the choice of this method for this study.

A multiple case design (Yin, 1989, p. 53) was used that gathered stories from trainees who were randomly selected. Rather than relying on a single case design where one particular trainee was investigated as single "case", many stories were deliberately heard involving numerous strategies and difficulties of transferring learning on the job. As was hoped, themes and categories emerged from a comparison of the stories, whilst recognition remained that each case had its own unique characteristics (Swanson-Kauffman, 1986, p. 65).

Yin (1989) argued, "using the case study method to gather data does not require statistical sampling because statistical generalization is not the aim of this method" (p. 38). Case studies should not be used to assess the incidence of phenomena (Yin, 1989, p. 55). They are not sampling units and should not be chosen for this reason (Yin, 1989, p. 38). Phenomenological studies, especially, should be concerned with the data and should, therefore, use non-probability sampling, with informants chosen because they have lived the experience. Sampling therefore was purposive, aimed at illuminating the richness of individual experience (Baker, Wuest, & Stem, 1992, p. 1358).

Sample and Procedures

The sample recruited for this research included trainees who had undertaken supervisory training programs at Hotel A and Hotel B. Two groups of trainees (Hotel A, N = 12 and Hotel B, N = 5) participated in the supervisory training programs. Of the 17 trainees, 2 were female and the rest were male. Their tenure as full-time employees varied from 2-5 years. This research was an exploratory study, thus it wanted to obtain a sample of specific individuals. According to Dane (1990), for exploratory or pilot projects the most appropriate method to recruit a sample is purposive sampling method. The criteria for selection were:

* having worked over 1 year at both hotels

* currently working as a supervisor or are being promoted to be a supervisor

* willing to participate in a research study

* being approved by the HR Director and the Training Manager for participation.

Prior to making contact with respondents, some key ethical issues had to be considered, such as the need for voluntary participation of informants, the paramount importance of the privacy of the respondent--especially given a workplace context, and assurance of confidentiality (Lipson, 1994).

This research employed two methods to collect data. The first was distribution of a questionnaire to obtain immediate feedback and the second involved interviewing the trainees.

Before distributing the questionnaire, the trainees were informed in detail about the purpose of the research and of the interview procedure. In addition, they were informed of their freedom to withdraw or refuse to participate. The questionnaire was divided into two sections. The first section used Likert scales to record trainees' reactions to the training program, specifically their views of the relevance of the program to their work situations. Questions were asked to determine respondents' biographical details, brief employment history, their goals for the program and their views of the program. The second part of the questionnaire asked about the trainees' experiences with their bosses since completing the supervisory training program. The questionnaires were left in the human resources (HR) office for the trainees to collect along with envelopes for their return. The completed questionnaires were returned in sealed envelopes to the human resources (HR) office for collection by the researcher. This method ensured the trainees' participation was entirely voluntary and confidential. All of the trainees involved in the training program under investigation were invited to participate.

To validate trainees' views as ascertained in the questionnaire, a semi-structured interview was conducted. This aimed to gather stories from the trainees after they had returned to their workplaces. The interview is one of the most important sources of case study information (Yin, 1989). Each trainee was interviewed twice for approximately 30 minutes to an hour on each occasion. The actual duration depended upon the trainee's condition at the time. Arranging a series of interviews allowed reflection on the transcripts of past interviews so as to gain as much interpretive insight as possible (van Manen, 1990, p. 99). The second interview proved to be extremely worthwhile as important information was volunteered, and trainees' answers were clarified, whilst emerging categories were validated. The interviews were recorded to give the fullest and most accurate record possible.

Data Analysis

The first part of the questionnaire was analysed using frequency counts. Question 1 to question 5 asked about the course offered and focused on its objectivity, relevance to the job, its organisation and the extent to which it met the individual's expectations. The following three questions highlighted the whole view of the course.

The next section aimed to investigate the transfer of learning on the job, and focused on the importance of immediate bosses encouraging their trainees to apply newly learnt skills and knowledge to the workplace. The trainees indicated whether or not they agreed or disagreed with each statement and could make further comments if they wished.

To analyse the semi-structured interviews, the researcher relied on transcription of the interviews as one source of raw data. The researcher read the transcripts and listened to the interviews many times, after which analysis could take place. After getting some sense of a whole, a consistent approach was selected. In this study inductive analysis was used, which means that "the patterns, themes and categories of analysis come for the data; they emerge out of the data rather than being decided prior to collection and analysis" (Patton, 1987, p. 150). A second meeting with the trainee aimed to validate and review what he or she had said and attempted to classify the trainee's comments. In this research, the paper did not dwell on categories but rather focused on the trainee's personal experience. The questions mainly focused on the trainee's day-to-day activities so the researcher was able to find out what strategies the trainees had applied to achieve better performance.