USING EAP CASE FILES TO DEMONSTRATE OUTCOMES 1
Using ExistingEmployeeAssistanceProgram
Case Files toDemonstrateOutcomes
Jodi M. Jacobson
Andrea L. Jones
Natalie Bowers
School of Social Work, University of Maryland
Abstract
Demonstrating outcomes and highlighting the value of employee assistance (EA) services are critical, especially in today’s difficult economic climate. This study reports findings from an exploratory study of Employee Assistance Program (EAP) outcomes using existing EAP case files from 20 different U.S. employers. Research questions examined the effects of EAP services as measured by three commonly used indices: the Level of Functioning Scales at Home and at Work, and the Global Assessment of Functioning. Significant changes in scores on all three measures were found at posttest, suggesting an overall positive affect from employee participation in EAP services. Recommendations for EAPs with little to no resources for research and limited experience with research are discussed.
Keywords: Employee Assistance Program, global assessment of functioning, outcomes
Acknowledgements
Thisresearch studywassupported bytheUniversityofMaryland,Baltimore,Schoolof SocialWorkthroughDesignatedResearchInitiativeFunds.Theauthorswouldliketoacknowledge thecontributionsand partnershipofFirstAdvantage Workplace Services,specifically, theircollaborationwiththeresearchteamandcommitmenttoadvancingtheEAP fieldthrough empirical study. Finally,the researchers would like to acknowledgethe contributions of Dr.TracyMcPhersonoftheGeorge Washington UniversityMedicalCenteronthisstudy.
Using ExistingEmployeeAssistanceProgramCase Files toDemonstrateOutcomes
Employees are not the only ones feeling pressured by their employers to “do more with less.” Employee Assistance Programs (EAPs) are feeling pressure from employers to provide high-quality services at a lower cost. At the same time, utilization of EAP services remains stable, if not elevated, due in part in response to the increased stress experienced by U.S. workers in today’s uncertain economy (Baltimore Sun, 2010; PompeThomas, 2008) and the trend for employees and their family members to seek EAP services paid for by their employer instead of traditional outpatient mental health services. Although not empirically studied, anecdotal reports from EAP leaders suggest a phenomena of “cost shifting” currently going on, resulting in additional employees presenting with more severe mental health and substance abuse problems that may not be appropriate for care within the EAP; however, due to barriers to access medical benefits, care may not be available to the individual and/or his or her family member.
In the past, EAPs have tried to respond to pressures to reduce costs bylimiting the number of allowable EAP sessions, decreasing the number of on-site events such as health fairs, educational workshops and screenings, substituting telephone counseling in place of face-to-face counseling, relying more on online and other technology, and lowering fees paid to EAP affiliates. These short-term responses only provide a “band-aid” solution to the larger problem. Without a well-developed, long-term strategy, EAPs may find themselves struggling to continue to be able to provide high-quality services. One alternative to reducing services and costs may be to demonstrate the value of EAP services currently provided and relate service delivery directly to return on investment (ROI).
One problem EAPs have historically faced is an inability to convert EAPclient data or case files into meaningful statistics that can be used to demonstrate value or outcomes related to direct EAP services. Reasons for not collecting and/or using data to demonstrate outcomes vary from EAP to EAP, but some of the more common reasons include lack of an appropriate and affordable standardized measure, lack or perceived lack of staff knowledge and/or time to analyze and report data, concerns about confidentiality, and negative feelings and perceptions about EAP evaluation (Jacobson & Attridge, 2010; Jacobson & Jones, 2010; Lennox, Sharar, & Burke, 2009). EAPs’ reliance on client satisfaction and other anecdotal evidence, rather than empirical outcomes based on standardized measures, will not provide a comprehensive picture to EAP customers highlighting possible outcomes achieved by EAPs (AttridgeAmaral,
2003; Csiernik, 2003; Csiernik, Hannah, & Pender, 2007; Hargrave & Hiatt,2004; Jacobson & Attridge, 2010; Jorgensen, 2007; Phillips, 2004; Sharar & Lennox, 2009). Demonstrating EAP outcomes with objective and standardized measures is the more difficult part of the EAP evaluation research.
There is evidence that EAPs, even with little to no resource for conductingresearch, can successfully demonstrate value of services with regard to important outcomes including but not limited to depression and other mental health problems, substance abuse, and family or relationship problems as they relate to business strategies, including productivity and performance (AttridgeAmaral, 2003; Burton et al., 2005; Hargrave, Hiatt, Alexander, & Shaffer, 2008; Harlow, 2006; McLeod, 2001; Rothermel et al., 2008; Wang, Simon, &Kessler, 2008; Yandrick, 1992). Some employers rely on their EAP as a primary provider of behavioral health support to minimize the negative effects of personal problems at the workplace (Steele, 1998). The business case exists supporting the value of EAPs and their role in helping to manage employees’ personal and behavioral health problems before they become personnel or productivity problems (AttridgeAmaral, 2003; Farris, 2003; Jorgensen, 2007).
Someofthemorecommon problems employees bringtotheEAPalsotendtobethemostcostlytoemployers. Theseincludedepression,substance abuse, and anxiety that can often exacerbate chronic health conditions (Lerneretal.,2004;Selvik,Stephenson,Plaza,Sugden,2004). Thenegative impactofsubstance abuse,depression,andotherbehavioralhealthproblems contribute toincreased absenteeismandpresenteeism, decreasedproductivity,sickleave,disabilityclaims,theft,accidents,andviolence(Hargrave etal.,2008;Pilette,2005;Selviketal.,2004;Substance Abuseand Mental HealthServicesAdministration,2006). Fortunately,thesetypesofbehavioral health problems are also some of the most preventableand modifiable chronic illnesses and can be managed with early intervention,access to appropriatecarewithinthecommunity,andsupported andsustainable behavioral change (Goetzel etal.,2007;Kahn,2008;KesslerStang,2006; Pelletier,2005;SlaymakerOwen, 2006).
Given valid and reliable measures, EAPscan demonstratethat they directlysupport theidentification ofproblems andemployeerecoveryfrom personal andbehavioralhealthproblems withirrefutabledata(Harlow,2006; Jacobson & Jones, 2010; Masi & Jacobson, 2003). EAPevaluation and outcomes measurementisachallenge formany EAPs, regardless ofsize. Without assistance from a statistician, resources to invest in a consultant andtimetocollectdata,EAPsoftenfindthemselvesinadifficultandstressful position toproduceempirically based outcomes fromdataabstracted from EAPcase filesin an effort to maintain existing contracts or compete for newbusiness.
The current study employed a pre- and posttest research design, usingsecondary data analysis methodology, to examine closed EAP case file from20 different U.S. employers. The primary research question was to find, what effect, if any, direct EAP services have on clients’ overall psychosocial functioning, as assessed by the Global Assessment of Functioning Scale (GAF) and two level-of-functioning questions: Level of Functioning at Work (LOF-W) and Level of Functioning at Home (LOF-H).
Method
Procedure
After receiving approval from the University’s Institutional Review Board (IRB), the researchers utilized secondary data to analyze EAP outcomes. This study employed a pre- and posttest single group research design that is commonly used in social science research to compare groups and/or measure change resulting from an experimental condition (DimitrovRumrill, 2003; Lennox, Sharar, Schmitz, & Goehner, 2010; Monette, Sullivan,DeJong, 2008).
Sample
All EAP cases files from the 20 U.S.-based companies identified for the study that were closed during 2007 had an equal chance to be selected for inclusion in the sample. The final sample included 572 closed EAP case files. Company classifications, based on the North American Industry Classification System (NAICS, 2007), included transportation and warehousing, scientific and technology services, finance and insurance, manufacturing, real estate, wholesale trade, educational services, food services, agriculture, forestry, fishing and hunting, health care and social services, administrative support, and waste management and remediation services. All employees from the20 companies and their covered dependents were eligible for EAP services.
Of the total sample, 303 (53.0%) were female, 267 (46.7%) were male, and 2 (0.3%) were not recorded. The majority (n=532, 93.0%) were employed full-time and were eligible for between one and three EAP sessions (n=360, 62.9%); the remainder were eligible for up to six EAP sessions (n=212, 37.1%). Most of the callers were “self-referred” to the EAP (n=512,89.5%) and had not previously used the EAP (n=399, 69.8%). Finally, themajority of callers (n=467, 81.6%) identified themselves as the employee with access to the EAP as part of their employee benefit package.
Of the EAP callers who were not seen face-to-face by the EAP (n=202;35.5% of all EAP callers), 90 callers declined additional services at the point of the initial telephone intake as their presenting problem was resolved after an initial supportive call and 112 did not show for their initial scheduled appointment. Results are based on the subsample of EAP callers who attended at least their initial face-to-face EAP assessment appointment and were therefore recoded as EAP clients (n=370, 64.7%).
The most common client age range was 30 to 39 years (n=102, 27.6%), followed by 40 to 49 years (n=101, 27.3%). Age ranged from 20[1]to older than 69 years. Slightly less than one half of the EAP clients were married (n=172, 46.5%); 20.5% (n=76) were single, never married; 9.7% (n=36) were divorced; 8.1% (n=30) were separated; 3.5% (n=13) had a significant other; 2.4% (n=9) were widowed; 0.5% (n=2) reported “other” for marital status; and 8.1% (n=30) declined to answer the question. The most common level of education completed was high school (n=91, 24.6%) followed closely by college (n=88, 23.8%). Whites represented the largest ethnic group (n=229, 61.9%), 12.4% (n=46) were African American, and 10.0% (n=37) were Hispanic/Latino. The most common salary ranges reported included$30,000 to $39,999 (n=67, 18.1%), followed by $20,000 to $29,999(n=54, 14.6%), and $40,000 to $49,999 (n=50, 13.5%). Approximately 13% (n=47) reported earning more than $200,000 annually. The average number of years of service for clients was 7.07 (SD=7.43). The most common employee job type reported was operations/labor (n=75, 20.3%), followed by management (n=53, 14.3%), and technology (n=43, 11.6%). The most common employee job level was nonsupervisory (n=169, 45.7%) followed by supervisory (n=63, 17.0%), and executive management (n=19, 5.1%). More than one half of the EAP services received were classified by the EAP as assessment and short-term counseling (n=198, 53.5%), and 45.9% (n=170) were classified as assessment and referral. The average number of counselor visits for each case was 2.84 (SD=1.42) with a range from 1 to 8 sessions.
Measures
Dependent Variables or Outcomes. The Global Assessment of Functioning Scale (GAF; American Psychiatric Association, 2000) is commonly used to assess psychiatric symptoms and functioning within mental health settings. The GAF score is obtained by assigning a clinician-rating on a 100-point scale ranging from 20 (dangerous) to 100 (fully functional). It is designed to measure three domains with one score: occupational, social, and psychological. Several researchers have examined the validity and reliability of the GAF and reported mixed results (Bacon, Collins, & Plake, 2002; Endicott, Spitzer, & Cohen, 1976; Jones, Thornicroft, Coffey, & Dunn, 1995; Niv, Cohen, Sullivan, & Young, 2007; Startup, Jackson, & Bendix, 2002; Vatnaland, Vatnaland, Friis, & Opjordsmoen,2007). Despite somewhat conflicting results with regard to validity andreliability, the GAF continues to be widely used to measure outcomes within the EA and broader mental health fields (Back-Tamburo, 2005; Jacobson & Jones, 2010; Murphy et al., 2009; Selvik et al., 2004; Stephenson et al.,2003). Specifically, researchers have reported better reliability estimates forthe GAF in research settings, then as compared to routine clinical environments (Vatnaland et al., 2007).
In a study of a Canadian EAP, Murphy et al. (2009) compared GAF scores at close of case for EAP clients seen face-to-face and those helped telephonically; no significant differences in scores were reported. Selvik et al. (2004) used the GAF, in addition to other outcome measures, to assess potential change in score from before to after EAP intervention. The average EAP client’s GAF score improved 10% post-EAP services. In an earlier review of EAP cases seen face-to-face compared to thosethat received telephone counseling, Stephenson et al. (2003) measured EAP utilization, number of EAP sessions, number of EAP cases assigned to various affiliate counselors, and client satisfaction. Results suggested improved GAF scores at case-closefor telephone counseling cases as compared to face-to-face cases (p=.05). Back-Tamburo (2005) compared client functioning measured by the GAF scale for EAP clients served in a stand-alone EAPs compared to an integrated EAP and work/life vendor. No significant differences on the postservices GAF score were detected (p<.05). In a survey of EAP owners, 40% of respondents who reported that their EAPs were currently using standardized measures to assess depression reported using the GAF as their indicator of depression and other general mental health functioning (Jacobson & Jones, 2010).
Level of Functioning (LOF) scales and indicesare commonly used in the human service field within outcome evaluations to assess activities of daily living and quality-of-life changes (Martin & Kettner, 1996). LOF scales often use Likert-type rating scales to measure functioning at work, home, and other settings where daily activity takes place. The current study used a single-item indicator to assess LOF at work (LOF-W) and LOF at home (LOF-H) with a5-point rating scale (1=very poor, 3=neutral, and 5=very good) as ratedby the EAP counselor. Several studies were identified that used LOF scales as a measure for evaluating EAP outcomes (Back-Tamburo, 2005; Greenwood, DeWeese, & Inscoe, 2005; Hargrave & Hiatt, 2004; S. M. Harris, Adams, Hill, Morgan, & Soliz, 2002; Masi & Jacobson, 2003; Selvik et al., 2004). Research suggests the existence of a positive relationship between direct EAP services and increased LOF among employees (Greenwood et al., 2005; S. M. Harris et al., 2002; Masi & Jacobson, 2003; Selvik et al., 2004).
Independent Variables. The researchers also explored the relationship of gender and age as they related to the three outcomes of interest.
Results
The researchers anticipated considerable missing data, as the database and EAP case file system was not designed initially for research or formal evaluative purposes. Of the 370 closed EAP case files, 50 case files had completed pre- and posttest GAF and LOF scores (14%). Although this rate is lower than what would typically be considered an acceptable response rate for research, it is common to see these rates within EAP practice. The current study will present exploratory results based on the limited number of case files; therefore, results should be interpreted cautiously. GAF scores ranged from 45 to
90 at intake (M=63.57, SD=10.62), and 50 to 90 at time of case close (M=69.51, SD=10.94). A series of dependent t tests were used to compare pre- and posttest scores for all three outcome measures. The changes in scores for all three measures were statistically significant (p < .005), suggesting an overall positive effect after participating in EAP services.
The average GAF score at intake was 63.57 out of possible 100 (SD=10.62), compared to an average score of 69.51 (SD=10.94) at case close,t(48)=-6.659, p < .005. At intake, clients scored an average of 3.47 out of5.0 (SD=1.14) on the LOF-W scale, compared to 3.96 (SD=1.00) at case close, t(46)= -5.402, p < .005. EAP clients scored an average of 2.76 out of5 (SD=1.03) on the LOF-H scale, compared to 3.57 (SD=1.10) at time ofcase close, t(48)= -7.144, p < .005. These preliminary findings suggest that client participation in face-to-face EAP services had a positive impact on the overall mental health functioning and overall level of functioning at work and at home.
Due to the exploratory nature of the data analysis, alpha was set at .10 to examine potential differences within gender and age. When gender was examined using independent samples t tests, no significant differences were found on the three clinicaloutcomes (p > .10). Age range was recoded into two categories: younger (39 years and younger, n=156, 42.2%) and older (40 years and older, n=212, 57.3%) (two files were missing data for age). A significant difference was detected for age as it related to LOF-H. Specifically, older EAP clients were more likely to have higher scores on LOF-H at case-close, t(47) = -1.881, p = .066. No other outcomes were significant (p > .10).
Discussion and Recommendations
Although overall results of the current study were positive, several limitations must be discussed. First, the lack of complete client records limited the researchers’ ability to utilize more complex statistical analyses and limited our ability to generalize beyond the current sample to the broader EA field. Additionally, the use of a pretest/posttest, one-group research design limits the researchers’ ability to control for other variables that may have influenced results, as there was no control group. Third, the EAP interventions are not standardized or manualized; therefore, what actually occurred during EAP in-person sessions varies from clinician to clinician. Without standardized treatment protocols or interventions, it is impossible to determine what aspects of the EAP service(s) contributed to positive results. Finally, bias in rating scores as determined EAP counselors for the GAF and the LOF scales should be considered. Without comparison of scores as rated by the client or other clinicians, it is difficult to determine their reliability and validity within the EAP setting.
There arealsoseveralstrengths related tothecurrent study.First,the willingness and opennessofthe EAPto collaborate with auniversity for research purposescannot beminimized. One requirementofworking with auniversity toconduct research isthat the university owns the data, and the researcher isfree topublish atwill. Thishas often been viewed aspotential threat byEAPs, similartoother work organizations, forfearthat negative resultswillbepublished. Lossofcontroloverdatahascontributed toreluctanceamongfor-profitEAPstopubliclysharedata. Thislackofaccess topublic databases and inabilitytopublish findingsdue totheproprietary nature ofthedatahavecontributedtostiflingthefield,preventing theEAP fieldfrommovingforward asaprofession (PompeSharar,2008;Roman,2007;Seidel, 2010;Tisone, 2008). The participating EAPshould be commended fortheircommitment tocontribute totheadvancementofempirical studywithintheEAPfield.
ThesecondmajorstrengthofthecurrentstudywasthefactthattheEAPwas utilizing the GAF,a standardizedmeasure to assess general mental health functioning. Theuseofstandardizedmeasures allowstheresultsto be comparedtoprior research. Anunexpectedoutcome from the current studythatlendsitselfasastrength toEAPpracticeistheparticipating EAP’s abilitytolearn from scrupulousreview ofclient data toaid research and initiatequalityimprovement,someofwhichhasalreadybeen implemented.
Overall,resultssuggest improvementon allthree outcome measures: GAF,LOF-W,and LOF-H. These findings areconsistent withother studies inthe EAPfieldusing similarmeasures (Greenwoodetal.,2005;Harlow,2006;Harrisetal.,2002;Selviketal.,2004). Changes inGAFscores pre- and post- EAPservices were consistent with findings reported by Selvik etal.(2004)GAFscorerangesatcaseclose(upper60s)wereconsistent with priorresearch (Back-Tamburo, 2005;Murphyetal.,2009;Selviketal.,2004). Two studies comparing GAFscores and different EAPservice modalities found nosignificantdifference inGAFscoresatthetimetheEAPcasewas closed based onmethod ofservicedelivery(Back-Tamburo, 2005;Murphy etal.,2009).