IsTrainingEffective?EvaluationoftransferfactorswithFETinthePublicAdministrationinSpain
Authors, organisation affiliation and position:
Dr. Pineda-Herrrero, Pilar
AssistantProfessor, AutonomousUniversity of Barcelona
BA.Espona, Berta
Researcher, AutonomousUniversity of Barcelona
MA.García, Natalia
Researcher, AutonomousUniversity of Barcelona
BA.Ciraso, Anna
Researcher, AutonomousUniversity of Barcelona
MA. Quesada-Pallares, Carla
PhD Student, AutonomousUniversity of Barcelona
Abstract
In this paper are presented the results of the ETAPE study, whose aim is to evaluate the effectiveness of continuing training in public administration in Spain. To evaluate that, the FET model was applied toshow which factors influence the transfer of training to the workplace. FET-ETAPE questionnaire (Pineda, Quesada & Ciraso, 2011) is one of the instruments of the methodology of this study, which combines quantitative data analysis with qualitative method in a non-simultaneous way, that is, in different phases. The most important results of this study are, on the one hand, a methodological and scientific progress that makes possible to predict learning of participants in training, the intent to transferof this learning and the application in the workplace; and, on the other hand, the factor that influences the most the transfer is orientation towards job’s requirements.
Keywords: continuing training, public administration, transfer of training, factors of transfer
1.Introduction
Currently,continuingtrainingplaysanimportantroleinhumanresourcepolicies,sinceitisakeystrategyfororganizationstoachievetheirobjectivesandtoenableworkers’professionaldevelopmentwithintheorganization.Todaycontinuouslearningisessential.
Inasituationofrapideconomic,political,andtechnologicalchanges,traininghasbeenincreasinglyconsideredasakeyelementinthedevelopmentofhumanresourcesmanagement,andinthechangesintheworkplace(Bossaert,2008).Companiesandinstitutionsinvestmanyresourcesintrainingtheiremployees,butrarelyknowtowhatextentthetrainingiseffective.Tosolvethisproblem,itisnecessarytoconductanevaluationofthetrainingtoverifytheeffectivenessandcostoftraining,intermsoftransferoflearning.
Therelevanceoftransferconsistsofevaluatingtowhatextenttheresultsobtainedduringtraining(learning)contributetothedevelopmentofprofessionalsintheorganization,i.e.theextenttowhichtrainingparticipantsapplytheacquiredlearningintheirworkplace.Forthisreason,thisstudyfocusesontheissueofevaluatingthetransferoftraining,specifically,onthecontinuingtrainingofworkersintheSpanishpublicadministration.
2.Theoreticalbackground
Theterm"transfer"wasfirstcoinedin1988(BaldwinandFord,1988).Theseauthorsdefineditasthedegreetowhichparticipantsapplytheknowledge,skills,andattitudesacquiredinthecontextofoccupationaltraining.Inorderforthetransfertakesplace,thelearnedbehaviourmustbegeneralizedtothecontextofworkandmustbemaintainedintheworkplaceforcertainperiodoftime(Pineda,2002).
Forthesereasons,theevaluationoftransferoftrainingbecomesaprioritytoknowtheeffectivenessoftrainingwithintheworkcontext.However,anexhaustiveprocessofevaluationoftransferrequiresmanyhumanandfinancialresourcesduetothedifficultyofmeasuringchangescausedbytrainingintheworkplace.
Becauseofthis,severalauthorsraisedthepossibilityofevaluatingtransferindirectly,throughthefactorsthatinfluencetheapplicabilityoflearningtotheworkplace.SuchisthecaseofthemodelsofBaldwinandFord(1988),Noe(1988),RouillerandGoldstein(1993),ThayerandTeachout(1995),Holton(1996,2005),BurkeandHutchins(2008),amongothers.
Fromtheabovemodels,themodelofHolton(version2005)standsoutasthetransferfactormodelthatincludesmoreelements.Itgroupsthemintothreedimensions:motivationalfactors,contextualfactors,andcapacityfactors.
ThestudiesoftheEFI-GIPEresearchteamfromtheAutonomousUniversityofBarcelonaarefocusedonevaluatingtheeffectivenessofcontinuingeducationfromtheperspectiveoftheimplementationoftrainingintheworkplace,thatis,fromtheperspectiveoftransfer.Thesestudieshavebeendonefromtwopointsofview(Pineda,QuesadaCiraso,2011):
- Directevaluationofthetransferoftraining.Itconsistsofcreatingspecificinstrumentstomeasuretheactualtransferoflearningintheworkplace(Pinedaetal.,2010;Moreno,Quesada,Pineda-Herrero,2010,Pinedaetal.,2011a,2011b).
- Indirectevaluationofthetransferoftraining.Itinvolvesthesearchforalternativestothemeasurementoftransferinordertoavoidthecostanddifficultyofdirectevaluation.Thediagnosisofthefactorsthatinfluencethetransferisthebestwaytodotheindirectassessment.Bydetectingbarriersandfacilitatorsoftransferinanorganization,wecanpredictwhetherornottherewillbetransferandintroducetheneededcorrections.Thus,factorsactaspredictorsoftransfer,allowinganindirectevaluationofit(Pinedaetal,2010;Pineda-Herrero,Quesada,Espona,2011;Quesada,2010,Pineda-Herrero,Quesada,Stoian,2011).
ThesestudiesallowedPineda,MorenoandQuesada(2010)todevelopatransferfactormodelfortheSpanishcontextinordertomeasurethevariablesthatcouldfacilitateorhinderthetransferoflearningintheworkplace.ItiscalledEvaluationofTrainingTransfer(ETF).Itwasadministeredto1044workersinSpain(Pineda,ed.2010).Fromitsvalidationsixfactorsemerged(seeFigure1).
Figure 1. The ETF factor model empirically validated. Source: AdaptedfromQuesada, EsponaPineda (2011)
TheresultsallowedthevalidationoftheinstrumentandtheanalysisofthefactorsthatdeterminethetransferintheSpanishcontext.Themodelalsoshowedasignificantrelationshipwithtransferandamedium-highpredictivepotential.
ThankstotheresultsoftheapplicationoftheETFin2010,theresearchofQuesada(2012)focusedondevelopingtheExploratoryModelofTransferInfluentialVariables(EMTIV)intrainingforthePublicAdministrationofCatalonia,itwasdevelopedandimplementedamorecomplexmodelontheFactorsforTransferEvaluationintheSpanishPublicAdministration,i.e.theFET-ETAPEmodel.
TheaimofthispaperistopresenttheresultsoftheETAPEstudyconductedduring2011inwhichtheFETmodelwasappliedtoevaluatethetransferofcontinuingtrainingreceivedbyworkersoftheSpanishpublicadministration.
3.Methodology
Thestudywasundertakenprimarilytoevaluatetheeffectivenessofcontinuingtraininginpublicadministrationinordertoimproveit.Thequestionthatguidedtheresearchwas:whatfactorsinfluencedthetransferoftrainingtotheworkplaceinthecontextoftheSpanishpublicadministration?Wehypothesizedthatthesefactorswererelatedtothreedimensions:participant,training,andorganization.
3.1.Method
Themethodologicalapproachwasmixedandnon-simultaneous,whichcombinedthecollectionofquantitativeandqualitativedataatdifferentstages.Inafirstqualitativephase,weperformedareviewoftheliteratureonfactorsthatmayinfluencethetransferoftraining.ThisfirstapproachwasvalidatedwiththefactorsresponsibleforthetrainingofeachofthefiveSpanishregionsthatparticipatedinthestudy.Thesecondphaseofthestudywasquantitativeandconsistedoftheapplicationoffourquestionnairestoasampleof1527employeesofthepublicadministrationofthefiveselectedSpanishregions.Thesequestionnaireswere:(1)theInitialQuestionnaireofSkills(CIP),whichevaluatedtheinitialdegreeofmasteryoftheskillstobetrained.Itwasappliedbeforestartingthetraining.(2)TheFactorsforTransferEvaluationQuestionnaire(FET-ETAPE),whichmeasuredthefactorsinfluencingthetransferoftrainingtotheworkplace.Itwasappliedjustbeforecompletionoftraining.(3)TheDeferredTransferQuestionnaire(CTd),whichwasappliedbetween2monthsand2½monthsaftercompletionoftrainingtoassessthedegreeofapplicationoflearningtotheworkplace.TheFinalQuestionnaireofSkills(CFC),whichwasappliedalongwiththeCTd,toassessthedegreeofimprovementinthemasteryofskillsoftrainingandtheuseoftheseskillsaftercompletingthetrainingprocess.
3.2.Sample
Todefinethesample,weconductedanon-probabilisticmultistagesampling(Hernández,Fernández-ColladoBaptista,2008)accordingtovariouscriteriaselectedfromthecharacteristicsofourresearch.Thefirstcriterionwasrelatedtothecontentoftraining.Thus,weselectedthreecontentareas:technological,legal,andsocialskills.Thesecondcriterionwastemporary,thatis,toselectthetrainingtobeevaluatedwechosetheonesheldduringthemonthsofApril,May,andJune2011.Fromhere,thethirdcriterionwastochoosetofollowthetrainingaccordingtoitsmodality:classroomoreLearning.
3.3.Techniques
TheFET-ETAPEquestionnaire(Pineda,QuesadaCiraso,2011)consistsof59items,whichmustbevaluedona5-pointLikertscale.Theitemsallowtoassessthe8factorsoftheFET-ETAPEmodelaswellastheachievedlearning(5items)andtheintenttotransfer(4items).Thefactorsatisfactionwithtrainingconsistsof8itemsandreferstotheparticipant’sreactiontothetrainingandtotheroleofthetrainer.(Tannenbaumetal.,1991,FordKraiger,1995;HolladayQuiñones,2003).Theaccountabilityconsistsof8itemsandreferstothedegreetowhichthemanageraskstheparticipantevidenceofchangesresultingfromtheimplementationoftraining(Baldwin,MagjukaLoher,1991,Fordetal.,1992,BurkeBaldwin,1999,Clarke,2002;Russ-Eft,2002;ChiaburuMarinova,2005).Ontheotherhand,thefactororientationtowardsjob’srequirements,consistingof7items,isdefinedastherelationshiporsimilarityoftrainingandmaterialswiththeneedsofparticipants’workpositionandthedevelopmentoftheircareers(BaldwinFord,1988;RouillerGoldstein,1993;FordKraiger,1995;Alligeretal.,1997;Ruonaetal.,2002,HolladayQuiñones,2003,LimMorris,2006).Thefactorenvironmentopportunitiesforapplicationiscomposedby10itemsandreferstoelementsexternaltotheparticipantthatmayinfluencetheapplicationofthelearningacquiredintrainingtotheworkplace(NoeSchmitt,1986,Tannenbaumetal.,1991,Holton,Bates,Ruona,2000;Clarke,2002;GaudineSaks,2004).Themotivationtotransfer(5items)referstotheinvolvementandthedesireofparticipantstoapplythetrainingtotheirworkplace(Fordetal.,1998,Holton,BatesRuona,2000).Theinternallocusofcontrol,consistingof5items,referstotheparticipant’sperceptionthathis/hersuccessorfailureinthetransferoftrainingtohis/herworkplacedependsonhim/her(NoeSchmitt,1986,Tannenbaumetal.1991,Saks,1995;ChiaburuMarinova,2005).Finally,thefactorspeer’ssupport(4items)andmanager’ssupport(3items)refertothedegreetowhichpartners(Xiao,1996;ChiaburuMarinova,2005)andmanagers(Fordetal.,1992,BurkeBaldwin,1999,Russ-Eft,2002;ChiaburuMarinova,2005),respectively,supporttheparticipantsduringtheprocessofapplicationoflearningtotheirworkplace.
Inthelastphaseofqualitativetype,weconductedtwogroupsofdicussion.Thefirstgroupwascomprisedofparticipantsintrainingfromjustoneregion.Thesecondgroupwascomprisedofthoseresponsiblefortrainingandmanagerswithexperienceintrainingfromalltheregions.Thesediscussiongroupsallowedustobetterinterprettheresultsobtainedwiththequestionnaires,afteradescriptiveanalysis,meancomparisonanalysisorANOVA,andmultipleregressionswithSPSS.
4.Results
TheresultspresentedbelowrefertothesampleG1ofthestudy,consistingoftrainingparticipantswhocompletedthequestionnaireFET-ETAPE(transferfactors)andtheCTd(deferredtransfer).Itwascomposedof1142workersfromtheSpanishpublicadministration.Asshowninthefollowingfigures,theyweremostlywomen(66%).Amongthejobtitles,skilledworkers(33%)andtechnicians(35%)arehighlighted.Themanagementpositionsgroupistheleastnumerous(3%).Thesamplewasbalancedaccordingtothecontentarea,withaslightmajorityofclassroomtrainingcomparedtotheeLearningmodality.Theaverageageofthesurveyedparticipantswas43years.
Figure 2. Distribution of participants by sex. Source: Authors’./ Figure 3. Distribution of participants by job tittle. Source: Authors’.
Figure 4. Distribution of participants by training’s content area. Source: Authors’.
/ Figure 5. Distribution of participants by training delivery. Source: Authors’.
Thefollowingfiguredisplaystheresultsofdescriptiveanalysisofthetransferfactors.Thedataareplacedina5-pointLikertscale.Valuesbelow2areconsideredabarriertotransfer.Valuesbetween2and3maybeariskofbarrier.Valuesbetween3and4representaweaktransferfacilitator.Finally,thevaluesabove4representastrongfacilitator.
Itisnotedthatonlytwofactors,satisfactionwithtrainingandmotivationtotransfer,obtainresultsthatplacethemasstrongfacilitatorsoftransfer.Meanwhile,accountability,withanaveragevalueof2.60,ispositionedasapotentialriskbarriertotransfer.Theotherfactorsareweakfacilitators.
Figure6.Descriptiveanalysisoftransferfactors.Source:Authors’
Figure7showsadescriptionoftheanalysedtrainingresults,i.e.,theintenttotransferandtheachievedlearning,whichwereevaluatedusingtheFET-ETAPEinstrument,andthetransfer perceptionfromthe CTdquestionnaire.Asnoted,thefirsttwovariablesgetahighscore,whiletheperceivedtransferiscloserto3withamedium-highresult.Thisresultcouldindicatethatalthoughparticipantshavelearnedduringtrainingandarewillingtotransfer,theydonotbelievetheyhaveappliedlearningintheirworkplacestotheextentonewouldexpect.
Figure7.Descriptiveanalysisoftrainingresults.Source:Authors’
Inordertoestablishcausalrelationshipsamongtransferfactorsandthehypotheticalindependentvariablesachievedlearningandintenttotransfer,towardstheperceivedtransfer,weusedsimpleandmultipleregressions.
First,andaftercheckingthatnoneoftheassumptionsoftheregressionwereinfringed,allvariableswereintroduced,consideringtheachievedlearningasdependentvariable.TheresultingadjustedR2ofthemodelwas0.57.However,itwasobservedthatsomevariableswerenotsignificantinT.Theanalysiswasrepeatedexcludingintenttotransfer,environmentopportunitiesforapplication,internallocusofcontrol,andmanager’ssupport.WeobtainedthemodelpresentedinTable8.
Table8.Multipleregressiontowardsachievedlearning.Source:Authors’
Independent variables / B / SE B / ß(Constant) / 0.36 / 0.10
Satisfaction withtraining / 0.47 / 0.02 / .50**
Accountability / -0.04 / 0.02 / -.04*
Orientation towards job’s requirements / 0.12 / 0.02 / .14**
Motivationto transfer / 0.33 / 0.03 / .26**
Peers’ support to transfer / 0.04 / 0.02 / .04*
Also,weconductedamultipleregressionoffactorsonthevariableintenttotransfer,introducingallthefactorsandthevariableachievedlearning.Amodelemergedthatexplained67.4%ofthevariance,butaccountability,peer’ssupport,manager’ssupport,andachievedlearningwerenotsignificantinT.Wethenproceededwithothermultipleregressions,excludingnon-significantvariables,andemergedamodelwithasatisfactoryadjustedR2(0.673)of5factors,asshowninthetablebelow.
Table9.Multipleregressionstowardsintenttotransfer.Source:Authors’
Independent variables / B / SE B / ß(Constant) / -0.15 / 0.09
Satisfaction withtraining / -0.05 / 0.02 / -.05**
Orientation towards job’s requirements / 0.19 / 0.02 / .22**
Environment opportunities for the application / 0.05 / 0.02 / .05**
Motivationto transfer / 0.69 / 0.02 / .54**
Internal locus of control / 0.19 / 0.02 / .24**
Wealsoperformedamultipleregressionofallfactorstowardstheperceivedtransfer.Amodelemergedthatexplained33%ofthevariance,butwefoundthat3factorswerenotsignificant:motivationtotransfer,peer’ssupport,andmanager’ssupport.Excludingthesefactors,weobtainedamodelwithanadjustedR2of0.329,asshowninTable10.Thisimpliesthatthedevelopedmodelcanexplainalmost33%ofthevarianceofthetransfer.Thispercentagemaybeconsideredappropriateinsocialsciences(intheabsenceofsimilarstudiesinourcontext,itisassumedthattheR2obtainedindicatesalargeeffect,followingtheadviceofCohen,1988).
Table10.Multipleregressionstowardsperceivedtransfer.Source:Authors’
Independent variables / B / SE B / ß(Constant) / 0.59 / 0.15
Satisfaction withtraining / 0.18 / 0.03 / .17**
Accountability / 0.08 / 0.03 / .08**
Orientation towards job’s requirements / 0.32 / 0.03 / .33**
Environment opportunities / 0.11 / 0.03 / .09**
Internal locus of control / 0.12 / 0.03 / .13**
Finally,weanalysedthesimpleregressionsamongtheindependentscalesachievedlearning,intenttotransfer,andperceivedtransfer,usingthelatterasdependentvariable.InTable11,wecanobservethatthethreerelationshipsweresignificantat99%.
Table11.Simpleregressionsamongindependentscales.Source:Authors’
Independent variables independientes / Dependent variables / ß / R2Achieved learning / Intent to transfer / ,49** / .24
Achieved learning / Perceived transfer / ,37** / .14
Intent to transfer / Perceived transfer / ,37** / .16
Byintegratingthemodelsandtheresultsofsimpleregressions,weobtainedthegeneralmodelpresentedbelow.Regardingthefactorsthatarespecificallyrelatedtothetransferresults,itappearsthattheorientationtowardsjob’srequirementshasahighercoefficient,andthereforehasagreaterweight.Theotherindependentscales,i.e.achievedlearningandintenttotransfer,alsohavearelativelyhighimpactinthetransfer.
Inthegeneralmodel,wenoticedallthefactorsthatemergedinthefactoranalysis,exceptmanager’ssupport.Itisobservedthatthefactorthathasahighercoefficient,andthereforeagreaterweight,istheorientationtowardsjob’srequirements.Italsohasastatisticallysignificantrelationshipwiththethreeseparatescalesandwiththesatisfactionwithtraining.Theotherindependentscales,i.e.achievedlearningandintenttotransfer,alsohavearelativelyhighimpactinthetransfer.
Figure 12. General model of transfer factors. Source: Authors’
Itisemphasizedthattheregressioncoefficientbetweensatisfactionwithtrainingandtheintenttotransferisnegative,whichindicatesaninverserelationship.Indiscussiongroupswithparticipantsintraining,theirmanagers,andthoseresponsiblefortraining,thisphenomenonwasrelatedtoparticipationintrainingduetopersonalinterestsandnottotheneedsoftheworkplaceortheorganization.Therefore,acoursemayobtainahighdegreeofsatisfaction.However,theremaybenointendedapplicationbecausethecontentsarenotlinkedtotheemploymentsituation.Thisphenomenonwouldoccurmainlyinthefreetraining,inwhichtheworkerisenrolledbyhisownwill,ratherthanoncoursesthatarestrategicforaparticularjob.
Therelationshipbetweenaccountabilityandachievedlearningisanotherinverseone.ThiscouldbeduetotheorganizationalcultureoftheSpanishpublicadministration,whereworkersperceiveaccountabilityasaprocessofcontrolandmonitoringonly.Thiswouldadverselyaffectthelearningofthetrainees.
Toanalysetheseresultsandmakesuggestionsforimprovement,itisusefultoconsidertheanalysisofcomparisonofmeansbetweenfactors,accordingtothecontentareaandthetypeoftraining.Tothisend,weconductedone-wayANOVAswiththeoverallsampleoftheETAPEstudy,whichconsistedof1527validcases.
Inthelocusofcontrolthereweresignificantdifferencesaccordingtothecontentarea(technological,legal,socialskills),F(2,1523)=23.89,p=.000.TheTukeytestindicatedthatthegroupofparticipantsfromthelegalfieldtraining(M=3.40,95%CI[3.61,3.74])obtainedsignificantlyhighervaluesonthisfactor,comparedwiththegroupofparticipantsinthetechnologicalareas(M=3.68,95%CI[3.61,3.74],p=.000),andinsocialskills(M=3.69,95%CI[3.62,3.75],p=.000).
Intheintenttotransfervariablewealsoobservedstatisticallysignificantdifferences,F(2,1522)=10.99,p=.00,withthesamepattern.TheGames-Howellpost-hoctestindicatesthatinparticipantsinthelegalarea(M=4.00,95%CI[3.94,4.06])thevaluesaresignificantlyhigherthanthoseinthetechnologicalarea(M=4.16,95%CI[4.11,4.21,p=.00])andinsocialskills(M=4.16,95%CI[4.11,4.22],p=.000).Thesedifferencescouldberelatedtothecharacteristicsoftrainingintheareaoflegalcontent,asreportedinthediscussiongroupwithparticipantsintraining.Often,legaltrainingisrelatedtoinnovationsintheadministrativeprocedurewhoseimplementationdoesnotdependononeself,butonallthoseinvolvedintheprocessandontheirwillingnesstopromotechange.Possibly,thelocusofcontrolmaybedisplacedmoreeasilyfromtheoutside,becausepeopledonotclearlyperceivetheirowncontrolinthetransfer.Thiscouldalsobeinfluencingtheintenttotransferandinhibitingit.
Inthecaseofaccountabilityontheimplementationoftraining,wealsodetectedsignificantdifferencesbetweenthethreecontentareas,F(2,1522)=10.60,p=.000.Instead,asindicatedbytheTukeytest,themeanintheareaoflegalcontent(M=2.72,95%CI[2.65,2.78])issignificantlyhigherthanthemeanoftrainingintechnology(M=2.51,95%CI[2.44,2.58],p=.000)andthemeanoftraininginsocialskills(M=2.57,95%CI[2.51,2.63],p=.003).Althoughthefactorinthiscontentareaislessthanascoreofthefacilitatorofthetransfer(2.72).Thesedifferencescouldberelatedtothereasonsleadingtoenrolintrainingcoursesand,ontheotherhand,thecharacteristicsofthevarioustraining,whichleadtoreceivedifferentconsiderationsbythetrainees’managers.Furthermore,asexpressedbysomepeopleinbothgroupsofdiscussion,socialskillstrainingseemlessconsistentwith“moreetherealknowledge”, workerstendtoenrolinthemduetopersonalinterestratherthanduetoaneedoftheworkplaceortheorganizationand,therefore,noaccountabilityprocessiscarriedout.
Wealsodetectedsignificantdifferencesinthefactorpeer’ssupport,accordingtothecontentarea,F(2,1522)=6.05,p=.002.TheGames-Howelltestinformsusthatinthecaseofsocialskills(M=3.23,95%CI[3.18,3.29])thetrainingshavelesssupportcomparedtothelegalcontentarea(M=3.39,95%CI[3.32,3.48],p=.001)andtotechnologicalarea(M=3.35,95%CI[3.28,3.41],p=.029).Thisdifferencemayberelatedtoanargumentputforwardbytheparticipantsinadiscussiongroup.Theyindicatedthatsocialskillsareanewtrainingfieldtowardswhichpeerscanprobablyhavemoreresistanceandmistrust,becauseitisunfamiliar.Inaddition,participantsintraining,managers,andtrainingtechnicianssaidthatcertainenvymightarise,especiallywhentraininginvolvestimeoffworkanditisnotperceivedasuseful.Thisaspectcanbeseenalsointhemanager’ssupportfactor,wheredifferencesweresignificant(F[2,1523]=5.23,p=.005)amonglegaltrainings(M=3.52,95%CI[3.44,3.60])inrelationtotheareaofsocialskills(M=3.33,95%CI[3.25,3.41],p=.002),accordingtotheapplicationoftheGames-Howelltest.
Alsobycontentarea,therearedifferencesinachievedlearning(F[2,1523]=19.46,p=.000).TheTukeytestindicatesthatthefactorissignificantlyhigherintrainingwithtechnologicalcontent(M=4.24,95%CI[4.19,4.29])thanthetrainingwithlegalcontent(M=4.07,95%CI[4.01,4.12],p=.000)andwithsocialskills(M=4.02,95%CI[3.96,4.07],p=.000).
Andfinally,wefindthatthemotivationtotransferis0.10higherintrainingsinthetechnologicalarea(M=4.32,95%CI[4.28,4.36])thanintrainingsinthelegalarea(M=4.22,95%CI[4.17,4.26],p=.002).TheTukeytestconfirmedthatthisdifferenceisstatisticallysignificant.
Regardingthetrainingmodalities,therearesignificantdifferencesinfactorssatisfactionwithtraining,accountability,orientationtowardsjob’srequirements,andenvironmentopportunitiesforapplication(seeTable13).AlldifferencesfavourtheclassroommodalityovereLearning.
Table13.Significantdifferencesinthefactorsbytypeoftraining(classroomoreLearning).Source:Authors’
Factor / F / df / df2 / pSatisfaction with training / 27,16 / 1 / 1524 / .000
Accountability / 5.56 / 1 / 1523 / .019
Orientation towards job’s requirements / 5,46 / 1 / 1524 / .020
Environment opportunities for the application / 7.00 / 1 / 1523 / .008
Thequalitativeinformationgatheredthroughdiscussiongroupsconfirmedthat,ingeneral,classroomtrainingoftenresultsinmorefavourablereactionsthaneLearning.Manyparticipantsagreedthatthelatter,althoughitismoreflexible,ismoredifficultbecauseitrequiresgreaterdisciplineandself-regulation.ELearningisoftenchosenforreasonsofbalancingfamilyandothercommitmentsthatwouldnotallowtomeettheattendancerequirementofclassroomtraining.However,itisassumedthattherearecomponentsoftherelationshipwiththetrainerorwithpeersthataffectthelearningprocess,andcannotbereplaced.Moreover,despitethegreaterflexibilityofonlinecourses,someparticipantsnotedthattheycouldcauseproblemsbecausetheparticipantdoesnothavetimeallottedfortraining.ThismaybeonereasonwhythesatisfactionwitheLearningislower.Afinalexplanationforthisresult,alsobasedonqualitativedata,isthatthereareworkerswhorequestspecificclassroomtrainingbutsincetherearenoplaces,theyareassignedtoaneLearningmodality.Thiscouldleadtodiscontentthatwouldinfluencetheresultsofthisfactor.
Anothersignificantdifference(Fstatistic)thatwedetectedbytypeoftrainingreferstotheaccountability,i.e.0.09lowerineLearning.Thisresultcanalsobeinterpretedwiththeinformationfromdiscussiongroups.Accordingtosomeparticipantsintraining,classroomcoursesaremoreevidentbecausetheworkersareabsentfromtheworkplace,anditiseasierthatwhentheycomebackamanageraskthemexplanationsonthetrainingandsubsequentlyonitsimplementation.However,inthecaseofeLearning,“theymightnotevenknowwhatisbeingdone”,whichexplainsthegenerallackofestablishedprocessesofaccountability.
Thefactororientationtowardsjob’srequirementsalsoshowssignificantdifferences(Fstatistic)bymodality,placingclassroomtraining0.09aboveeLearning.Accordingtotheperceptionsofparticipantsindiscussiongroups,thisresultcouldberelatedtothenumberofstudentsinthecourses.Inclassroomcoursesthegroupissmaller,soitwouldbeeasiertopersonalizethetraining,adjustingittotheneedsofparticipantsandtheirspecificjobs.
Finally,wedetectasignificantdifference(Welchstatistic)inthefactorenvironmentopportunitiesfortheapplication,whichhadameanlower(0.09)ineLearningtrainings.
5.Discussion
Theevaluationofthetransferoftrainingaimstomeasureitseffectivenessintheimprovementoftheperformanceandprofessionalfunctioningoforganizations.ThisstudydemonstratesthattheFETinstrumentallowstomeasurethefactorsofthetransferoftraininginacompleteandexhaustiveway,andprovidesanindirectmeasurementoftransferthroughitspredictivecapacity.
TheapplicationoftheFETinstrumentintheSpanishpublicadministrationshowsthatthepossibilitiesoftransferarepositivegiventhehighsatisfactionandmotivationoftheparticipants.Theachievedlearningandtheintenttotransferalsohavehighscores.However,therearefactorsthatcanclearlybeimproved.Theseincludethefactoraccountability,whichappearsasabarrier.ThismaybeduetothelimitedtraditionintheSpanishworkculture(especiallyinthepublicenvironment)ofaccountingforinvestmentsintraining.
Analyseshaveshownthatthefactorsidentifiedcanreadilypredictlearningofparticipantsintraining,theintenttotransfertheselearning,anditsapplicationintheworkplace.Thisresultconstitutesanimportantmethodologicalandscientificprogress.Giventhecomplexityofmeasuringtrainingresultsdirectlyintheworkplace,thepossibilityofindirectassessmentthroughfactorsprovidesarevolutionarynewapproach.TheFETmodelmakesitpossiblesomethingsofarunattainable:theevaluationoftransferwithasingleeasytoapplyquestionnaire,whichalsoprovidesverysuggestiveresultsforimprovingtheeffectivenessoftraining.
Inthepredictivemodelwefound,whichexplains33%ofthevarianceofthetransferperceivedbytheparticipants,emergekeyfactorsforachievingeffectivetraining:orientationtowardsjob’srequirements,whichisthefactorwithmoreweightintransfer;satisfactionwithtraining;workers’perceivedcontroloftheimplementationoflearningintheworkplace(internallocusofcontrol);environmentopportunitiesforapplication;andaccountabilitytothemanagerontheresultsoftrainingatwork.Resultsallowtostatethattrainingwillbeeffective,thatis,itwillproducetransfertotheworkplace,ifithastheaforementionedfactorswithhighscores.
Thegeneralmodelwehavedevelopedsuggeststhatinordertopredicttheresultsoftransferandactontheitemsthatcanbeimproved,wedonothavetoevaluatethelearningachievedbyparticipantsoftrainingandtheirintenttoapplyitintheirworkplace.Instead,theevaluationofthefactorsthemselvescanpredicttheextenttowhichpublicemployeesapplytheskillsacquiredintheirprofessionalperformance.Thispredictionoftransferhasimportantimplicationsbecauseitallowstomakeimprovementstoincreasethetransparencyandeffectivenessoftraining.Thefactorthatmostinfluencesthetransferisorientationtowardsjob’srequirements,whichreferstothelinkingoftrainingwiththeparticipant’semploymentsituation; this factor can be improved with a rigorous analysis of participants’ needs for each training, or choosing materials, exercises and activities related with labour reality of workers.
Aneffectivecontinuingtrainingisthebesthelpthatanyprofessionalcanhaveonthecurrentcontextofeconomicuncertaintyandjobloss.Guidelinesofthiskind,whichcomefromtheevaluationoftheresultsobtainedwiththeFETinstrument,cangreatlyhelporganizationstoimprovetheeffectivenessoftheirtraining.Thesesarethegoalsofourresearch:assessingtraininginaneasyandaccessibleway,andhelpingorganizationstoimprovetheeffectivenessoftheirtraining.
6. References
Alliger, G.M., et al. (1997). A meta-analysis of the relations among training criteria.Personnel Psychology 50(2), 341-358.
Baldwin, T. T., & Ford, J. K. (1988).Transfer of training: A review and directions for future research. Personnel Psychology, 41(1), 63-105.
Baldwin, T.T., Magjuka, R.J., & Loher, B.T. (1991). The perils of participation: Effects of choice of training on trainee motivation and learning. Personnel Psychology, 44, 51-65.
Bossaert, D. (coord.) (2008). Training and human resource development in the European Union member states.New practices and innovative trends.Maastricht: European Institute of Public Administration.
Burke, L. A., & Hutchins, H. M. (2008).A study of best practices in training transfer and proposed model of transfer.Human Resource Development Quarterly, 19(2), 107-128.
Burke, L.A., & Baldwin, T.T. (1999).Workforce training transfer: A Study of the effect of relapse prevention training and transfer. Human Resource Management, 38(3), 227-243.
Chiaburu, D.S., & Marinova, S.V. (2005). What predicts skill transfer? An exploratory study of goal orientation, training self-efficacy, and organizational supports.International Journal of Training and Development 9(2), 110-123.
Clarke, N. (2002). Job/work environment factors influencing training effectiveness within a human service agency: Some indicative support for Baldwin and Fords’ transfer climate construct. International Journal of Training and Development 6(3), 146–162.
Cohen, J, (1988). Statistical Power Analysis for the Behavioral Sciences. 2nd. edit., Hillsdale, N.J.: Erlbaum.
Ford, J.K., & Kraiger, K. (1995).The application of cognitive constructs and principles to the instructional systems model of training: Implications for needs assessment, design, and transfer. In C.L. Cooper & I.T. Robertson, ed., International Review of Industrial and Organizational Psychology 10, 1–48.
Ford, J.K., Quiñones, M.A., Sego, D.J., & Sorra, J.S. (1992). Factors affecting the opportunity to perform trained tasks on the job.Personnel Psychology 45(3), 511-527.
Ford, J.K., Smith, E.M., Weissbein, D.A., Gully, S.M., & Salas, E. (1998).Relationships of goal orientation, metacognitive activity and practice strategies with learning outcomes and transfer.Journal of Applied Psychology 83, 218–233.
Gaudine, A.P., & Saks, A.M. (2004).A longitudinal quasi-experiment on the effects of posttraining transfer interventions.Human Resource Development Quarterly 15(1), 57-76.
Hernández, R., Fernández-Collado, C & Baptista, P. (2008). Metodología de la investigación, 4ª ed.México: McGraw-Hill.
Holladay, C.L., & Quiñones, M.A. (2003).Practice variability and transfer of training: The role of self-efficacy generality. Journal of Applied Psychology 88(6), 1094–1103.
Holton, E. F., Bates, R. A., & Ruona, W. E. A. (2000).Development of a generalized learning transfer system inventory.Human Resource Development Quarterly, 11(4), 333-360.
Holton, E. F.,III, & Kirkpatrick, D. L. (1996). The flawed four-level evaluation model.Human Resource Development Quarterly, 7(1), 5-29.
Holton, E. F.,III. (2005). Holton's evaluation model: New evidence and construct elaborations. Advances in Developing Human Resources, 7(37), 37-54.
Lim, D.H., & Morris, M.L. (2006).Influence of trainee characteristics, instructional satisfaction, and organizational climate on perceived learning and training transfer. Human Resource Development Quarterly 17(1), 85-115.
Noe, R. A. (1996). Trainees' attributes and attitudes: Neglected influences on training effectiveness. Academy of Management Review, 11(4), 736-749.
Noe, R.A., & Schmitt, N. (1986).The influence of trainee attitudes on training effectiveness: Test of a model. PersonnelPsychology 39, 497-523.
Pineda, P. (2002). Gestión de la formación en las organizaciones.Barcelona: Ariel.
Pineda, P., Quesada, C., & Ciraso, A. (2011). Evaluating Training Effectiveness: results of the FET Model in the Public Administration in Spain.7th International Conference on Researching Work and Learning. Shanghai: Normal University of China.
Pineda, P., Úcar, X., Moreno, V. & Belvis, E, (2011a). Evaluation of teachers’ continuing training in the early childhood education. Teacher Development, 15, 205-219.
Pineda-Herreno, P., Belvis. E., Moreno, V., Duran-Bellonch, M. & Úcar, X. (2011b). Evaluation of training effectiveness in the Spanish health sector.Journal of Workplace Learning, 23.
Pineda-Herrero, P., Quesada, C. & Stoian, A. (2011). Evaluating the e-learning efficacy in Spain: a diagnosis of learning transfer factors affecting e-learning.Procedia-Social and Behavioral Sciences Journal, 30.
Pineda-Herrero, P., Quesada, C., & Moreno, M.V. (2010).The ETF, a new tool for Evaluating Training Transfer in Spain.11th International Conference on HumanResource Development: Human Resource Development in the Era of GlobalMobility. Pécs: University of Pécs.
Quesada, C., Pineda-Herrero, P., & Espona, B. (2011). Evaluating the efficiency of the leadership training programs in Spain.Procedia-Social and Behavioral Sciences Journal, 30, 2194 – 2198.
Rouiller, J. Z., & Goldstein, I. L. (1993).The relationship between organizational transfer climate and positive transfer of training.Human Resource Development Quarterly 4(4), 377-390.
Ruona, W. E. A., Leimbach, M., Holton, E. F.,III, & Bates, R. (2002). The relationship between learner utility reactions and predicted learning transfer among trainees. International Journal of Training and Development, 6(4), 218-228.
Russ-Eft, D. (2002). A typology of training design and work environment factors affecting workplace learning and transfer. Human Resource Development Review, 1(1), 45-65.
Saks, A.M. (1995). Longitudinal field investigation of the moderating and mediating effects of self-efficacy on the relationship between training and newcomer adjustment.Journal of Applied Psychology 80, 211–225.
Tannenbaum, S.I., Mathieu, J.E., Salas, E., & Cannon-Bowers, J. A. (1991).Meeting trainees’ expectations: The influence of training fulfillment on the development of commitment, self-efficacy, and motivation. Journal of Applied Psychology 76(6), 759–769.
Thayer, P. W., & Teachout, M. S. (1995). A climate for transfer model.Armstrong Laboratory/Human Resources.
Xiao, J. (1996). The relationship between organizational factors and the transfer of training in the electronics industry in Shenzchen, China.Human ResourceDevelopmentQuarterly 7, 55-73.