Andrea Herschman and Sarah Forde

Andrea Herschman and Sarah Forde

Tunaweza (Kiswahili: we can do it!): Measuring the Impact of Sport on Girls’ Life Skills

“Sport in development” has recently been integrated into the international development agenda, and numerous sport initiatives have been established throughout the developing world. These programs consider sport a catalyst for social change[1], and a forum for learning valuable life skills, such as confidence, teamwork, and leadership. These skills are assumed to carry into other aspects of life, promoting positive outcomes, including: sustainable peace, gender equity, HIV/AIDS and reproductive health education, and economic development. While aid and development agencies advocate the use of sport to promote life skills, and the ability of these skills to change behavior and create positive development outcomes, such claims lack empirical evidence and support.

This paper presents a method for measuring changes in life skills developed by Moving the Goalposts (Kilifi) (MTG a sport-in-development program in Kilifi, Kenya. MTG uses soccer to improve the life skills of adolescent girls and young women. The aim of this paper is to contribute to the understanding of sport as a tool for improving life skills. However, in order to demonstrate the ability of sport to affect these skills it is necessary first to establish a means of measuring changes in life skills. The sport-in-development literature has only just begun to identify procedures for assessing the impact of sport in developing country contexts (Coalter 2006). MTG’s life skills assessment method provides a model for participatory monitoring and evaluating (PM&E) in other sport-in-development programs. This method, and a discussion of its successes and shortcomings, contributes to the development of a means to better understand the effects of sport on life skills.

We begin, in Section 1, with a discussion of the relationship between sport and life skills. In Section 2, we introduce MTG and present the hypotheses developed to test the impact of its life skills program. In Section 3 we present the method for testing these hypotheses. We focus on the participatory approach, context specific indicators, and the development and implementation of the life skills survey tool. In section 4, we discuss the benefits and challenges associated with this method, the potential for replication, and offer conclusions and suggestions for future research.

I. Can sport improve life skills of youth in the developing world?

There is a longstanding belief that sport teaches valuable lessons that carry over into other aspects of life, benefiting both individuals and society. This attitude is pervasive within the western world, and two decades of critical examination by sports scholars have done little to erode this basic assumption (Gatz, Messner et al. 2000). Numerous intervention programs in the developed world targeting at-risk youth are based on the connection between sport, life improvement and changed behavior, and ultimately, positive social outcomes. Sport has been used to attempt to prevent crime (Coalter, Allison et al. 2000; Nichols 2004), reduce anti-social behavior (Coalter 1996; Morris, Sallybanks et al. 2003), rehabilitate at risk adolescents (Andrews and Andrews 2003), and reduce the risk of teenage pregnancy (Sabo and Melnick 1996).

These studies have assumed that sports participation produces desired results indirectly through intermediate outcomes. Studies have presented results suggesting that sport improves emotional and cognitive skills such as self-esteem and problem solving (Morris, Sallybanks et al. 2003). Other scholars have proposed hypotheses regarding the mechanisms leading to positive social outcomes including: direct aversion from offending; an increased sense of control over one’s life; personal fitness and improved mental health; improved social skills; and the development of pro-social values (Morris, Sallybanks et al. 2003).

The idea that sport influences behavior has spilled over into the international arena, and sport-in-development programs have adopted sport as a catalyst for social change. These programs have applied sport to a different set of global challenges including sustainable peace, gender equity, and HIV/AIDS prevention (U.N. 2003). The development literature shares the belief that sports participation engenders development results through intermediate outcomes. (Picture 1 depicts this general relationship). These intermediate outcomes are frequently grouped together and referred to as “life skills.” For example, The United Nations advocates that sports programs can teach essential life skills necessary for HIV/AIDS protection and prevention. In this case, the intermediate outcome responsible is the sense of ownership and respect for an individual’s body developed by adolescent girls through sports participation. These “life skills” encourage them to delay sexual activity. Similarly, Sabo and Melnick have proposed a link between sport and decreased risk of pregnancy among adolescent girls. They hypothesize that elevated self-esteem derived from participation in sport influences sexual decision-making and encourages the use of contraceptives.

The term “life skills” has been criticized for being ambiguous and unclear (Boler and Aggleton 2005). The concept is defined in various ways by different organizations. The lack of a clear definition of the concept and its components has generated confusion and difficulty in studying the causes or outcomes of such skills. Broadly, life skills can be defined as,

“a large group of psycho-social and interpersonal skills which can help people make informed decisions, communicate effectively, and develop coping and self-management skills that may help them lead a healthy and productive life. Life skills may be directed toward personal actions and actions toward others, as well as actions to change the surrounding environment to make it conducive to health.” (UNICEF 2007)

This definition highlights the intermediate role of life skills in modifying an individual’s behavior. Implicit in this model of decision-making is the assumption that individuals will utilize improved life skills in risky or challenging situations. Such individual changes are believed to lead to changes in individual behavior, and ultimately to positive development outcomes affecting the overall community. In this paper we focus narrowly on the relationship between sport and life skills for two reasons. First, changes in individual behavior such as the decision to have protected rather than unprotected sex, and subsequent development outcomes resulting from sports participation, such as gender equity and sustainable peace, are difficult to measure. Because of this limitation, we redirect our efforts to the intermediate causal mechanism of life skills, which we believe is methodologically feasible to measure and assess. Second, by concentrating on this link we hope to contribute theoretical clarity to the study of sport as a tool for development. To see the link between sports participation and changes in behavior and other development outcomes, the intermediate mechanism must first be understood.

2. MTG and the life skills monitoring and evaluation program

MTG was established in 2001 in Kilifi, Kenya. MTG is a unique sport-in-development program in its specific promotion of sport for girls. The majority of sport-in-development projects begin with boys as the program focus and only incorporate girls as a secondary program component. MTG has adopted a “sports plus” approach (Coalter 2006) in which the organization uses football and related activities to address a number of broader social problems affecting young women in Kilifi. These issues include early school dropout, teenage pregnancy, early marriage, HIV/AIDS, poor leadership and decision-making skills, and limited economic opportunities. MTG addresses these issues by utilizing soccer participation to improve life skills in disadvantaged girls and young women. MTG defines life skills specifically as: confidence, leadership, self-esteem, team-work, and organization.

MTG’s core value is girls’ participation. This encompasses participation in decision making, organizing and leading activities, and as peer educators, coaches and referees. This provides opportunities for some of the most disadvantaged girls (such as those who have a limited level of formal education) to take on tasks and responsibilities that they would otherwise not be exposed to. As a small, low budget, community based organization MTG recognized the capacity building potential of initiating a life-skills M&E project in order to assess the effectiveness of their program. A team of 6 girls volunteered to help design and implement the M&E project. Through their involvement in the process they received training in the development and administration of questionnaires. These processes built their capacity to contribute effectively to community development[2]. In addition to capacity building, MTG wanted to broaden its impact by contributing to the evidence base of sport-in-development programs in Africa.

In line with the aims of MTG’s strategic plan, it developed the following four hypotheses in order to test the effect of sport on the life skills of their participants:

H1: An individual’s life skills will improve over the course of their participation in MTG.

H2: The life skills of MTG participants are higher than those of non-participants.

H3: Individuals involved in MTG for many years will have higher life skills compared with those involved for a short time.

H4: Individuals who are closest to the ‘core’ of MTG’s programs, and participate regularly, will have higher life skills than girls who participate less.

MTG has utilized multiple qualitative methods for addressing the hypotheses above. These include an annual “My Story” competition in which players express, through stories, poetry, rap, or video, how MTG has affected their lives, and an in-depth “Life Stories” project, documenting the lives of a small group of players over time. However, this paper focuses exclusively on the organization’s development of a context-specific quantitative method for testing these hypotheses. Quantitative analysis of the effects of sport on life skills was desirable for two reasons. First, quantitative analysis provides a check on MTG’s qualitative findings. Statistical analysis enables MTG to use a multiple methods approach to better understand how sport affects girls in Kilifi. Second, quantitative analysis allows MTG to draw inferences about the effect of sport on life skills to its overall population of participants. While in-depth qualitative data provides rich information on a small group of participants, it does not allow for generalizations to be made about the effect of sport on life skills throughout the MTG population more generally[3]. The following sections document the process of developing a context-specific quantitative measure, its capacity building elements, and its potential for low cost replicability in other sport-in-development projects .

3. Methods

The participatory monitoring and evaluation (PM&E) approach emphasizes participant involvement in defining what will be measured and how a phenomenon will be assessed (Bradley, Mayfield et al. 2002). The phases of the life skills M&E initiative were designed with this participatory emphasis in mind. The first phase identified context-specific indicators of life skills in adolescent girls in Kilifi, Kenya. The organization then developed a survey designed to measure changes in the indicators. Finally, the survey was implemented and analyzed.

Phase 1: Developing context-specific life skills indicators[4]

Inkeeping with the organization’s core value of girls’ participation, the primary goal of the first phase of the life skills M&E initiative was to use a participatory approach to identify context-specific indicators of life-skills. The participants in this phase of the project were girls from within the organization representing different ages and educational backgrounds. All of the girls played on an MTG team, and many also volunteered for the organization as coaches, referees, and girls’ committee members[5]. The primary techniques utilized were: key informant interviews, semi-structured interviews, focus group discussions (FGDs), participatory learning and appraisal (PLA) techniques, and observation. An outside researcher, two research assistants selected from within MTG, and a female translator collected data.

FGDs were conducted with players from MTG, and generated a list of life skills indicators. FGD participants then analyzed what each of the life skills “looked like,” shared examples of how and where they would demonstrate these attributes, and identified specific changes they had experienced while playing football. They were also encouraged to discuss their ideas about measuring changes in these skills. Semi-structured interviews were conducted with families and other adults to triangulate the data from FGDs.

FGDs yielded numerous indicators. The research team, together with a 15 member girls’ committee from MTG, employed participatory learning and appraisal techniques (PLA) to refine this list. The list was scored, voted on and pair-wise ranked to determine which indicators best reflected each of the life skills. The end result was a list of two to three indicators for each life-skill. For each of the life skills, definitions and sub-indicators were developed. Definitions helped to clarify the exact meaning of each indicator. Sub-indicators identified specific characteristics associated with each life skill. For example, the group designated “courage,” “not being shy,” and “believing in yourself” as indicators for confidence. Examples of the sub-indicators included: speaking in public, and initiating a conversation with an elder.

The process for arriving at the list of life skills indicators was a time consuming process. Reaching consensus on definitions and indicators of such complex concepts was difficult. The process was further complicated by language. All of the girls in MTG speak Kiswahili as their main language of communication outside of the home (in the home, more often than not, KiMijikenda is spoken). The outside researcher (a Masters student from the UK) spoke only English. However this issue was mitigated by the diverse use of participatory methods and the mentoring skills of the outside researcher, which made the process both empowering and enjoyable for the girls involved. As one participant stated,

‘I wasn’t shy, I had a lot of confidence and I felt very happy. I was able to do something that other girls said I couldn’t do… I want to do more FGDs to develop my skills’ (informal evaluation of process)

Phase 2: Life skills survey development

The outcome of Phase 1 was a list of indicators of life skills. During Phase 2, these indicators were utilized to develop a researcher-administered survey to measure changes in life skills[6]. As a capacity building measure the M&E team took the lead, with support from an outside researcher, in formulating a list of questions from the previously identified indicators. Some of the questions created by the M&E team were open-ended (i.e. “what would you do if…?”), and not conducive to quantitative analysis. After discussion with the M&E team the researcher converted the open-ended questions into a quantifiable survey. Once revised, the M&E team members discussed, evaluated, and revised each question, selecting questions they believed best captured and assessed the relevant life skill.

This process produced a long list of questions. To reduce the number of questions, the M&E team took the questionnaire home and independently selected one question to eliminate from each life skill category. The team then voted for which question they thought should be removed. The question receiving the most votes was discarded. The M&E survey tool consisted of the remaining questions. In general, the M&E team voted to eliminate questions that required respondents to recognize that they possess the trait in question. Questions were included in which the relevant life skill was acknowledged by others. For example, the M&E team voted to discard the following question from the leadership section of the survey: “I feel that I am good at understanding the problems of others.” In contrast, a question reading, “If one of my teammates has a problem I am one of the first people they come to for advice,” was retained. This subtle difference highlights the importance of context specific indicators. It was important that questions recognized the social implications of the life skill, rather than the individual’s possession of them.

The final survey was translated from English into Kiswahili and then back translated[7].

Phase 3: Survey Implementation

The participatory nature of the life skills M&E initiative continued during survey implementation, as the M&E team was responsible for administering the survey. Because of MTG’s emphasis on participatory monitoring and evaluation, the potential for capacity building within the organization, and the costs of employing outside individuals, program participants were selected to administer surveys. Because of the specific skills required to administer surveys, MTG provided a day-long workshop and training session on survey-administration.

The outside researcher, and a volunteer research field worker from Kenya Medical Research Institute (KEMRI) conducted the training. Utilizing a local volunteer, who works in research, was effective in teaching the survey administration methods. The volunteer conducted the training in Kiswahili. Due to the complex and subtle nature of the concepts discussed it was important to conduct the training in the language that the team was most familiar with. As a young woman from the area, the volunteer related closely to the girls on the team. As a role model, she provided additional motivation to the M&E group.