Effectiveness of the Completion Bonus 1
Effectiveness of Completion Bonuses
for Achievement in Adult Education
Dr. Mary Ziegler
Olga Ebert
Gail Cope
Center for Literacy Studies
The University of Tennessee
July 2001
Table of Contents
List of Tables
List of Figures
Executive Summary
Introduction
Review of the Literature
Motivation Theory and Research
Adult Learners and Motivation
Bonus and Incentive Programs
Research Methodology
Summary of the 1999 Study
Sample and Data Collection
Assumptions
Data sources and types
Context and Definitions
Use of grade level
Federal changes to levels after completion of the 1999 study
Number of hours needed to gain a grade level
Limitations
Variability in number of subjects
Time period for participants’ learning gains
Differences in baseline characteristics of samples
Data Analysis
Findings
Findings From Participant Records Pre- and Post- Bonus
Demographic information comparing the two groups
Comparison of days needed to achieve learning gains
Comparison of number who achieved learning gains
Comparison of traditional ABEAE students in the same time period
Differences between urban and rural participants
Number of bonuses awarded
Findings From Interviews and Focus Groups
Focus groups
ABEAE administrator interviews
Summary of Findings
Conclusions and Recommendations
Appendix A: Participant Instrument for Focus Groups
Appendix B: Instrument for Administrative Interviews
References
List of Tables
Table 1. Demographic Information
Table 2. Families First: Comparison of Days to Achieve Learning Gains or Make Progress
Table 3. Number of Families First Participant Who Achieved Learning Gains
Table 4. Achievement of Traditional ABEAE Students
Table 5. Comparison Between ABEAE Students and Families First (FF) Participants
List of Figures
Figure 1. Enrollment periods for comparison groups.
Executive Summary
In March 2000, the Tennessee Department of Human Services (DHS) introduced the Completion Bonus Program for participants in Families First. “This bonus program was designed and implemented to encourage the completion of educational and employment outcomes that will lead to self-sufficiency and career advancement” (Metcalf, 2000).
The Tennessee Department of Human Services requested that the Center for Literacy Studies at The University of Tennessee examine the effectiveness of the completion bonus for improved performance in adult education (AE). The design of the study included a replication of a prior study conducted in 1999 before the Completion Bonus Program was implemented to compare the performance of Families First participants (pre-bonus group) before and (post-bonus group) after the implementation of the bonus program.
The same 16 programs, 2 urban and the rest rural or semi-rural, located in the three grand regions of the state, participated in both studies. Data from 955 participant records collected before and after the implementation of the completion bonus were compared to determine whether the bonus program influenced the performance of Families First participants in basic education classes. It is important to note that, although according to the DHS administrative data, overall Families First enrollment slightly increased (approximately 8% between summer 1999 and spring 2001), the overall enrollment of Families First participants attending AE did not change according to the data from the survey of AE program administrators that was a part of this study.The main study findings are summarized as follows:
1.The median number of days needed to achieve a learning gain was the same for the pre- and post-bonus groups. The bonus may not be an incentive to achieve at a faster rate.
2.Compared to the pre-bonus group, significantly more Families First participants in the post-bonus group made learning gains. The bonus may be an incentive for participants to persist until they have achieved a learning gain.
3.Substantially more Families First participants made learning gains after the introduction of the bonus compared to traditional AE students in the same programs who were not eligible for the bonus.
4.In the post-bonus period, 112 Families First participants have achieved the 6th-grade level in math and reading; in the comparable pre-bonus time period, 52 people made this gain in the same programs (115% increase). Overall Families First enrollment in AE had not increased.
5.In the post-bonus period, 123 Families First participants have achieved the 9th-grade level in math and reading; in the comparable pre-bonus time period, 27 people made this gain in the same programs (356% increase). Overall Families First enrollment in AE had not increased.
6.In the post-bonus period, 189 Families First participants passed the GED test; in the comparable pre-bonus time period, 81 people from the same programs passed the GED(133% increase). Overall Families First enrollment in AE had not increased.
7.Most participants and administrators (or teachers) perceive that the bonus is a motivator to persist in achieving educational goals, and their perceptions correspond to the quantitative data collected from program records.
While this study showed that the bonus did not make a difference in the number of days a participant took to make progress, it did report a significant increase in the number of people who advanced a level or passed the GED examination. The benefit of the incentive may not be to increase the rate at which participants make progress but to attract people to persist who might have otherwise dropped out. While the completion bonus is only one factor involved in making progress in adult basic education, it may prove to be a very important motivational incentive for those who otherwise might not have persisted in achieving their goal.
Introduction
In 1996, the Tennessee Legislature established Families First, a welfare-reform initiative administered by the Department of Human Services (DHS). Families First provides cash grants, education, job training, child care, employment assistance, and transitional benefits to prepare eligible adults for lasting employment and independence from welfare. A report on the characteristics of Families First participants who received welfare benefits shows that more than half do not have a high school diploma or a GED (General Educational Development) credential. Without adequate basic education, employment prospects are narrowed considerably. Individuals who lack a high school diploma but earn a GED have more potential for higher wages when compared to those who do not have a high school credential (Bos, 1995; Davis, 1998; Boudett, Murnane, and Rubenstein, 2000). In addition to higher earnings, the GED increases access to postsecondary education and training programs that lead to career advancement and even higher earnings over time. Improving basic skills and receiving a GED play a vital role in increasing the earnings potential of welfare recipients.
Because of the importance of basic skills for successful employment and employment retention, Families First legislation provided the opportunity for participants who were functioning below the 9th grade to enroll in adult education classes for 20 hours a week. Individuals who function below the 9.0 grade level in reading or math are offered the opportunity to enter AE for 20 hours per week. If the individual chooses to enter AE for 20 hours per week, he/she will be exempt from additional work components and from time limits. This exemption will last until such time as he/she tests at the 9.0 grade level or higher in both reading and math or stops going to AE. An individual who chooses not to attend AE classes will not be exempted from time limits and will be subject to normal work-related requirements and sanctions. After passing the 9.0 level, a Families First participants becomes subject to time limits and a 40 hour work requirement, 20 of which can be AE. Classes, for those participants who choose to attend, provide instruction in basic skills, practice in common workforce skills, and preparation for the GED test.
In March 2000, the Tennessee Department of Human Services (DHS) introduced the Completion Bonus Program. “This bonus program was designed and implemented to encourage the completion of educational and employment outcomes that will lead to self-sufficiency and career advancement” (Metcalf, 2000). To motivate participants to engage in educational and work activities, the program offers cash bonuses for achieving certain thresholds, such as improving basic skills, earning a GED or high school diploma, receiving a college degree, completing employment preparation classes, obtaining and keeping a job, and leaving Families First due to earnings.
Encouraging performance with the incentive of a cash bonus is rare in most adult basic education programs across the country:. “Programs that decide to use incentives will be testing new ground and will have the opportunity to examine whether this approach has promise and is worth of further, more rigorous evaluation” (Hill & Pavetti, 2000, p. iv). The purpose of this study was to evaluate the effectiveness of the bonus program for Families First participants who attend adult basic education classes and determine whether “this approach has promise.”
The Center for Literacy Studies (CLS) at The University of Tennessee, at the request of the Department of Human Services (DHS), conducted a study to evaluate the effectiveness of the bonus. The research question that guided the study was “Does the bonus program influence the progress made by Families First participants in adult education.” This study addressed the question by examining the performance of Families First participants in adult education prior to and after the implementation of the bonus program.
The following report has four main sections in addition to the Introduction. The second section reviews the literature on the use of bonuses or rewards for performance. The research methodology, selection of the sample, data collection, instruments, context, and limitations of the study are in the third section, Research Methodology. Findings are presented in the fourth section, and conclusions and recommendations are in the last section of this paper.
Review of the Literature
The purpose of the present research was to better understand the relationship between external rewards and the achievement of educational outcomes. The literature review examines the theories of motivation and incentive systems.
Motivation Theory and Research
Research on motivation has been conducted in numerous disciplines, including education, psychology, and business. A comprehensive body of literature has emerged on motivation theory and its application in classrooms, social services agencies, and workplace settings. Research and practice have made important distinctions between the concepts of intrinsic motivation and extrinsic motivation (Cameron & Pierce, 1994; Deci & Ryan, 1985; Herzberg, 1989; Wlodkowski, 1999). Individuals have intrinsic motivation when they engage in an activity for its own sake and experience interest and enjoyment. They have extrinsic motivation when they perform a behavior or activity to attain a separate reward that is not part of the activity itself. The issue of intrinsic versus extrinsic motivation has been a hotly debated topic.
According to some researchers, one widely cited study, conducted by Cameron and Pierce (1994), examined reward type (verbal and monetary), reward expectancy (whether or not the reward was expected), and reward contingency (based on participating in an activity, completing an activity, or performance). Extrinsic rewards can have both positive and negative effects on intrinsic motivation. Their study suggested that extrinsic rewards do not adversely impact intrinsic motivation and concluded that incentive systems are an effective method for motivating students in classroom settings.
Studies by Deci, Ryan, and Koestner (1999a, 1999b) presented a different perspective on the effect of extrinsic rewards on intrinsic motivation. Many of the studies they cited (e.g., Rummel & Feinberg, 1988; Tang & Hall, 1995; Wiersma, 1992) concluded that there is “support for the proposition that expected tangible rewards made contingent upon doing, completing, or excelling at an interesting activity undermine the intrinsic motivation for that activity” (p. 627). The meta-analyses by Deci, Ryan, and Koestner included aggregated results from prior research that included participants from different age levels: elementary age students; high school students; college students; and adult learners.
The negative effect of extrinsic rewards has been referred to as the undermining effect (i.e., extrinsic rewards reduce the intrinsic motivation to engage in a behavior or activity). These contradictory results on the role of incentives in education leave gaps in the knowledge base and lead to further questions about the use of incentives with adult learners, especially those who may be reluctant to revisit education after a long absence.
In a recent article, Deci, Koestner, Ryan, & Cameron (2001) examined the interactive effects of intrinsic and non-monetary extrinsic rewards:
To summarize, research indicates that verbal rewards (i.e. positive feedback) tend to have an enhancing effect on intrinsic motivation; however, verbal rewards are less likely to have a positive effect for children than for older individuals. Furthermore, verbal rewards can even have a negative effect on intrinsic motivation if the interpersonal context within which they are administered is controlling rather than informational.
The relationship of monetary rewards and extrinsic motivation has not been studied in depth, particularly in relation to adult learners. One of the goals of the present study is to shed light on this issue.
Adult Learners and Motivation
Wlodkowski (1999) described motivation as a concept that explains why people think and behave as they do: a human process for directing energy to accomplish a goal. While motivation is recognized as a crucial element in adult learning, it is a concept that is difficult to directly observe and measure. Indicators of motivation have been extensively studied and researched in adult education, social psychology, and industrial-organizational psychology. These indicators include effort, perseverance, persistence, performance, and completion of activities or tasks. Motivation is, therefore, inferred from self-reports of effort, task persistence, and goal accomplishment.
Persistence in adult education programs has been a topic frequently studied (Beder, 1999; Comings, Parrella, & Soricone, 1999; Fingeret, 1985; Mezirow, Darkenwald, & Knox, 1975) because it is an enduring problem. Persistence relies on one’s motivation to continue on a given course despite obstacles. “Adults must make active decisions to participate in each class session and often must overcome significant barriers to attend classes” (Comings et al., 2000, p. 1). Most of the persistence studies have been conducted with adults who volunteered to attend adult basic education classes. The authors were unable to find any study that examined the motivation to persist among participants in a welfare-reform program who must attend adult education to be eligible for benefits. Persisting with education can be very difficult for adults who are parents, have limited resources, lack access to reliable transportation, or do not have dependable child care. Incentives may play a role in increasing one’s motivation to persist.
Bonus and Incentive Programs
The Families First completion bonus program is a type of incentive program. One of the main types of incentive programs described in compensation management literature is the bonus system. In workplace settings, formal bonus plans are the most common type of incentive plan and have been used traditionally to compensate upper-management employees or to reward sales personnel. However, use of incentive plans has increased during the past several years to include all levels of employees. The formal bonus is typically paid on a yearly basis; and the amount of the bonus is based on a combination of individual job performance and the business performance of the department, the company, or both.
A recent report issued by Mathematica Policy Research (Hill & Pavetti, 2000) describes the use of incentives in promoting job retention and advancement among disadvantaged workers. The report drew on research and field studies primarily from the private sector and the performance-improvement industry. Incentives were often used as a performance-improvement tool; and cash was the most common type of incentive, with gift certificates and merchandise ranking second and third, respectively (Nolon & Alonzo, 2000).
The welfare system in the United States and some employment and training programs have used incentives in some form since the late 1960s. Many state welfare plans, for example, have implemented the Earned Income Tax Credit (EITC), earned income disregards, and financial incentives to move participants into the workforce. The Minnesota Family Investment Plan (MFIP), the Self-Sufficiency Program (SSP) in Canada, and the New Hope Program (NHP) in Wisconsin all offer a variety of incentives; and evaluation results indicate that these programs are successful in increasing employment rates, increasing earnings of participants, and encouraging participants to move off welfare and into the labor force.
Many researchers of incentive and reward systems have noted the contradictory and inconsistent nature of the findings and have attempted to identify conditions under which these programs are effective or ineffective. For instance, Cameron and Pierce (1994) examined factors such as reward type (verbal or tangible), reward expectancy (whether the reward is or is not expected), and reward contingency. The various reward contingencies that have been studied include rewards that are given for participating in an activity, rewards given for completing an activity, and rewards given for attaining a specific level of performance. Their findings are summarized as follows:
- Verbal praise and feedback enhance a person’s intrinsic interest.
- Rewards can have a positive impact on motivation when they are given after first stating clear performance standards.
- Teachers have no reason to resist implementing incentive systems in classrooms.
Kohn (1996, 1999) took a very different perspective on the use of reward and incentive programs and noted that the use of incentives may result in substandard work performance. He added that some incentive programs seem to work in the short term but that such programs are ineffective in the long term. Kohn challenged the behavioral perspective that has traditionally been part of the U.S. educational system as well as compensation strategies in the workplace. In addition, he referred to at least 70 studies that concluded that rewards or incentives undermine interest in the task or behavior itself. Nelson (1994) explained that shortcomings of incentive programs are not necessarily the incentive itself but are often the result of poor planning and management practices. He added that, for an incentive program to be effective, it should be well integrated with performance-management strategies. Nelson’s suggestions have practical implications for caseworkers and adult education teachers who work closely with Families First participants. If individuals are to improve their performance, they need better individual performance planning in which goals, expectations, and rewards are mutually established; regular coaching to support and encourage them in reaching their goals; and timely feedback about progress.