Comparison of Alternative Employment Services for Persons with Serious Mental Illness: Effects on Patient-Reported Mental Health Functional Status, Substance Use, and Arrests.
Salkever D, Steinwachs DM, Abrams M, Gibbons B, Baier K, Stuart EA, Skinner EA, Wu AW, Salzberg C.
ABSTRACT
Objective:To evaluate the effect of initiation of evidence-basedsupported employment services (Individual Placement and Support-IPS) for persons with serious mental illnesses (SMI) and qualifying for state or federal disability on patient-reported mental health functional status, substance use, and arrests.
Data Sources:Maryland Medicaid data was matched to data from the Division of Rehabilitative Services and the Maryland Public Mental Health System to identify the study groups. Data from Maryland Medicaid, Maryland Mental Hygiene Administration (MHA) and the Maryland Division of Rehabilitation Services (DORS) were used to select study subjects, to identify initiation dates of employment services that occurred between Sept 1, 2006 and Dec 31, 2007, and to assign each study subject to a treatment group. Data from MHA's Outcomes Measurement System (OMS) were used to measure baseline levels of the self-reported outcome measures and follow-up levels of these same measures.
Study Design: An intent-to-treat design is used to compare three groups of Medicaid persons with SMI: those initiating evidence-based IPS (placing people in competitive employment with support services) meeting fidelity criteria (IPS-F), those initiating supported employment services without evidence of fidelity to IPS (SE-NF), and those initiating traditional vocational services, emphasizing training that is expected to lead to competitive employment (TVS). Propensity score weights are used in comparing the three groups on a number of baseline variables from the Medicaid claims data measured over the 12 months prior to the assigned initiation date of employment services for each study subject. Outcome variables were measured from OMS data for follow-up periods of 12-months, 24-months, and 36-months following each subject’s date of employment services initiation.
Data Extraction Methods:Data sets were provided by the Maryland Medicaid Program, the Maryland DORS and the Maryland MHA's Public Mental Health System.
Principal Findings: Results based on a variety of regression methods with inverse-probability propensity weights, showed inconsistent patterns of differences among the three treatment groups. Failure to reject null hypotheses of no difference, however, must be viewed as tentative due mainly to the relatively small number of study subjects with available OMS dependent variable data in each of the three study groups.
Conclusions: The tentative conclusion is that the 3 study groups did not differ on these patient-reported outcomes. Further assessment with additional data is, however, indicated because of sample size and OMS data availability issues.
Keywords: Supported employment; individual Placement and Support (IPS); Patient-reported mental health outcomes.
Acknowledgements:This research was supported by Contract No. HHSA290201000009I from the Agency for Healthcare Research and Quality, US Department of Health and Human Services, as part of the Developing Evidence to Inform Decisions about Effectiveness (DEcIDE) Program.
INTRODUCTION
Nationally, it is estimated that persons with serious mental illnesses (SMI) and serious functional impairment comprise 4.1% of U.S. adults (SAMHSA, 2013). Recent data also suggest that over 80% of persons with schizophrenia being treated in the community are not employed (Salkever, et al., 2007). Moreover, employment is cited as a goal by almost 2/3 of all persons with SMI in the U.S. public mental health system (Bond and Drake, 2014; Bedell et al. 1998; Frounfelker et al. 2011; McQuilkenet al. 2003; Mueser, Salyers and Mueser, 2001; Ramsay et al. 2011; Rogers et al. 1991; Watkins et al. 2011; Woltmann 2009). In contrast studies show that only about 15% report any current paid employment (Bond and Drake, 2014; Lindamer et al. 2003; Pandiani and Leno 2012; Perkins and Rinaldi 2002; Rosenheck et al. 2006; Salkever et al. 2007).
Given this large gap between those wanting to work and those actually working, increasing paid employment has been viewed as a principal recovery goal for rehabilitation programs for those with SMI. Research shows that Individual Placement and Support (IPS), an evidence-based supported employment intervention, can achieve competitive employment positions for half or more of persons with SMI desiring work (Bond, Drake, and Becker, 2008). In randomized trials of IPS vs. traditional rehabilitation programs that are not vocationally oriented, IPS has shown strong positive effects in helping patients to gain paid (mainly competitive) employment (Marshall et al., 2014). As Salkever (2010, 2013) has noted, comparisons with more traditional alternative vocationally-oriented services, such as an enhanced vocational rehabilitation (EVR) (Drake et al., 1999) or a diversified placement approach (DPA) (Bond et al. 2007) show significantly higher competitive employment rates for IPS but similar overall paid employment rates (including agency jobs and other non-competitive paid employment) for EVR and DPA.
In contrast to clear results about IPS effectiveness in achieving competitive employment, IPS effectiveness in achieving nonvocational outcomes has been the subject of debate in the recent literature. A recent article by two of the leading authorities in the field (Bond and Drake, 2014) forcefully argues that IPS does have positive nonvocational effects stemming from the combination of (1) positive IPS effects on employment and (2) the positive effects of employment on nonvocational outcomes. Their argument is as follows:
IPS is a highly effective approach to vocational rehabilitation…(Becker et al. 2011)….systematic reviews conclude that IPS enhances vocational outcomes (Bond 2004; Bond et al. 2008; Burns et al. 2007; Crowther et al. 2001; Dixon et al. 2010; Twamley et al. 2003). About two-thirds of IPS participants succeed in competitive employment…(Bond et al. 2012a) and sustaining employment for years (Becker et al. 2007b; Salyers et al. 2004)….
People who obtain competitive employment through IPS enhance their income, self-esteem, quality of life, social inclusion, and control of symptoms (Bond et al. 2001; Burns et al. 2009; Kukla et al.2012; Mueser et al. 1997;Turner et al. 2012). These enhancements to well-being persist at 10-year follow-ups (Becker et al. 2007b; Salyers et al. 2004). People with SMI often report that IPS is good treatment and central to their recovery (Bailey 1998;Becker et al. 2007b; Strickler et al. 2009)….employment leads to decreased mental health costs (Bond et al. 1995; Burns et al. 2009;Clark 1998; Henry et al. 2004; Latimer 2001; Perkins et al. 2005; Rogers et al. 1995; Schneider et al. 2009). Long-term cost reductions appear to be even greater (Bush et al. 2009).
An alternative view is expressed in a recent article by Kukla and Bond (2013), who report on a two-year randomized trial of an IPS program vs. a DPA program. The nonvocational outcomes studied pertained to symptoms, psychiatric hospitalizations, quality of life, and social networks. They summarize their results and conclusion as follows:
Although the total sample showed improvement in several nonvocational domains over time, there were largely no differences between groups in nonvocational outcomes at follow-up or in their rates of improvement over time…Participation in supported employment alone is not sufficient to positively impact most nonvocational outcomes in people with severe mental illness.
To further reinforce Kukla and Bond’s (2013) alternative view that questions IPS effects on these nonvocational outcomes, we have previously noted (Salkever 2010 and 2013) that evidence of a positive contemporaneous association between employment status and these nonvocational measures should not be interpreted as a causal influence of employment because of the strong likelihood of unmeasured selection factors that are positively correlated with both employment and the nonvocational measures. This same concern about inferring causation is in fact explicitly noted in Kukla et al. (2012), Bond et al. (2001), Burns et al. (2009), and Mueser et al. (1997), all being papers cited by Bond and Drake in support of their argument for the nonvocational effects of IPS.
Given the divergence of views (e.g., between Kukla and Bond (2013) and Bond and Drake (2014)) about the effects of IPS on nonvocational outcomes, we undertook our own analyses that expanded the range of the research in several ways. First, it presented comparisons between certified fidelity-compliant IPS vs. the alternative “treatments” of (1) SE that was not certified as fidelity compliant and (2) a range of other “traditional” vocational rehabilitation services provide by a state vocational rehabilitation agency. Second, these analyses involved new outcome measures. In the present analysis, we study patient-reported outcome measures from a new statewide Outcomes Measurement System (OMS) used in the state of Maryland for all patients served under Maryland’s Public Mental Health System-managed specialty mental health care program. In a companion study, state Medicaid claims data are used to measure IPS vs. alternative treatment effects on measures of continuity and coordination of treatment services, testing the hypothesis that continuity and coordination of these services for persons with severe mental illness may be influenced by IPS efforts to coordinate vocational services with other mental health treatment services (Steinwachs, et al., 2014).
Traditional employment services in Maryland differ from the IPS supported employment approach in that they emphasize training and non-competitive job experiences (e.g., enclave employment) prior to or instead of placement in a competitive employment environment.
In 2002, the Maryland Mental Hygiene Administration implemented a statewide program to provide evidence-based employment services for all persons with SMI wanting to work, largely replacing traditional employment training and placement programs. For persons with SMI, competitive employment has been recognized as key outcome of the recovery process (President’s New Freedom Commission on Mental Health (2003); US Surgeon General (1999); National Institute of Mental Health (1999)). We hypothesized that the IPS employment intervention would have desirable effects on non-employment outcomes. Specifically, we expect that the IPS employment intervention would have positive effects on individuals’ self-reported functional status, and would reduce the frequency of negative outcomes such as substance use and arrests.
In this study, comparisons are made among consumers meeting SMI and continuous Medicaid enrollment criteria who entered either an IPS program meeting established fidelity standards (IPS-F), a supported employment program without fidelity (SE-NF) or a traditional employment services program (TVS). We hypothesized that IPS-F would be associated with improved coordination of services as compared to TVS and possibly similar to SE-NF employment program outcomes. IPS-F is expected to:
§ Increase the positivity of each of 5 patient-reported ratings of their functional status with respect to 1) participating in meaningful activities, 2) taking care of personal needs, 3) coping with problems, 4) doing things the patient wants to do, 5) reducing the perceived burden of mental illness symptoms.
§ Reduce the frequency of self-reported alcohol use, drug use, and substance abuse.
§ Reduce the frequency of self-reported arrests.
BACKGROUND
Individual Placement and Support (IPS) is a version of supported employment that uses a “place and train” approach (Wehman and Moon, 1988) in contrast to traditional practices in which extensive pre-vocational training is provided prior to job placement (“train and place” model). The supported employment model was adapted to meet the needs of persons with SMI and multiple randomized controlled trials have established its efficacy in improvement employment outcomes (Bond, Drake and Becker, 2008). In nine RCTs conducted in eight states (including Maryland), IPS employment rates ranged from 27% to >75%, compared to controls that achieved 7% - 40% IPS. A recent Cochrane Review of 14 RCTs in the U.S. and elsewhere compared IPS with other programs (largely emphasizing training prior to work placement) and focused on the employment outcomes of time to obtain competitive paid job sand duration of employment. IPS programs consistently outperformed traditional employment services (Kinoshita, 2013; Marshall et al., 2014)).
According to the Dartmouth Psychiatric Research Center (2011), IPS is based on 8 principles: (1) involvement in competitive employment from the outset instead of placement in a sheltered or non-competitive work setting, (2) all consumers (the person with the SMI) desiring to work are eligible for IPS, (3) rapid job search (placement frequently within a month), (4) integration of mental health and employment services, (5) attention to consumer preference in the job search, (6) time-unlimited individualized job supports, and (7) personalized benefits counseling, and (8) systematic job development.
DATA AND METHODS
Study Population - The sample selection criteria identified persons with serious mental illness (SMI) diagnoses and state or federal certified disability as identified in the Medicaid enrollment files with at least 10 months per year of Medicaid enrollment over a 4-5 year period, 2 years prior and 3 years post initiation of an employment services intervention. Persons were eligible for inclusion if their first employment services contact occurred between September 1, 2006, and December 31, 2007, which marked the beginning of statewide implementation of an Outcomes Measurement System (OMS) that is used by Maryland’s Public Mental Health System (PMHS) to track the progress of the patients that they serve. Persons were included in the study if they met all the following selection criteria. The figures in parentheses show the numbers remaining after completion of the step.
1. Had a serious mental illness diagnosis in calendar years 2006 or 2007 (n=131,820)
2. Was 20-63 years of age between 9/1/2006 and 12/31/2007) (n=79,759)
3. Eligible for disability benefits according to state or federal criteria during the period 7/1/2006 to 12/31/2007 (n=35,253)
4. Was not a Medicare enrollee at initiation of employment services (n=22,556), given that Medicare data were not available for this investigation
5. Had at least 10 months of Medicaid enrollment in each 12 month period two years before and two years after the employment services initiation period, 9/1/2006 and 12/31/2007 (n=16,321)
6. Had a matching DORS or PMHS record indicating initiation of employment services between 9/1/2006 and 12/31/2007 (n=618)
7. Did not receive employment services in the year before the initiation period and lived in Maryland during the same period (n=433)
Serious mental illness was defined as the presence of at least one of the following International Classification of Diseases Version 9 (ICD-9) diagnostic codes, as recorded in the Medicaid data:
295.xx Schizophrenic disorders
296.xx Episodic mood disorders (mania, bipolar, major depression)
297.xx Delusional Disorders
298.xx Other Nonorganic Psychoses
299.8x, 299.9xOther pervasive developmental disorders
300.xx Anxiety states