Adolescent Development and Participation Section, Programme Division

Terms of Reference

Consultancy: Documentation, monitoring and evaluation of collaboration and pilot testing of the Adolescent Kit

Adolescent Development and Participation Section, Programme Division

Time Frame: 182 work days between 15 September 2014 and 31 May 2015

1. Background

The Adolescent Development and Participation (ADAP) Sectionwithin UNICEF’s Programme Division works to advance the rights of adolescentsacross all UNICEF supported programmes and policies. The section promotes cross-sectoral programming for adolescent girls and boys that builds on their assets and strengths, while addressing their unique vulnerabilities and risks.

Adolescents, defined by the United Nations as people ages 10-19,[1]comprise almost half of the children in UNICEF’s mandate. However, there remain significant gaps in UNICEF’s global programming resources for this key age group, especially those affected by emergencies. Adolescents are profoundly affected by armed conflict, natural disasters and other crises, facing grave risks to their lives, safety and development. They have the ability and the will to rebuild their communities and to be resources for peace and stability, and the right be active in shaping their lives, communities and futures. Yet in past years UNICEF has not had ready, accessible, relevant resources to reach and engage adolescents in these contexts, and support them in developing to their full capacity and pursue their priorities. Especially lacking have been resources addressed to the unique needs of girls and boys, those with disabilities, and those from marginalized sub-groups. Recognizing adolescents as a key stakeholders and potential partners, some UNICEF country programmes have initiated creative, effective strategies to work for and with adolescents. However, useful lessons and good practices from these experiences have not always been gathered and shared between countries and regions in accessible, practical formats that staff and partners can use in programming, especially in high-pressure environments such as emergencies.

To build on these possibilities and address these gaps, UNICEF's Adolescent Development and Participation (ADAP) Section is developing an “Adolescent Kit,” which is a package of guidance, resources and supplies to reach, engage and promote the rights of adolescents before, during and after emergencies and in other low-resource contexts. The kit supportsapproachesandactivities that can be used in UNICEF’s child protection, education, and youth programmes and partnerships, including Child-Friendly Spaces, schools and education programmes (both formal and nonformal), youth centres and community-based protection programmes. The approachesoffered in the kit focus on finding andreaching children ages 10-18 who are hardest to reach, working in flexible spaces and involving adolescents as decisionmakers throughout the programme cycle.The activities utilize arts, creative expression, cultural traditions, problem solving, experimentation and innovation, especially through projects which adolescents design and carry out based on their own interests and priorities.These activities and approaches are drawn from successful programmes implemented by UNICEF and partners in a wide range of global contexts.

The Adolescent Kit is designed to be flexible and adaptable to the needs and priorities of adolescents in different contexts. It applies a cross-sectoral approach, is designed for use through a variety of different

programmes, and is aligned to standards, practices and indicators used in both the education and child protection sectors. It is harmonized with UNICEF’s Core Commitments for Children, appropriate and adaptable for the preparedness, response and recovery phase of emergency, as well as development contexts. The kit is also appropriate and adaptable for younger adolescents (10-14) and older adolescents (15-18 and older). Guidance tools in the kit will support users in selecting and adapting activities and approaches for use in these different contexts.

The kit reflects UNICEF's evolving approach to peacebuilding, with a focus on building adolescents’ resilience and their capacities to contribute to peace.The activities and approaches supported by the kit are designed to address ‘competency domains’ or ‘outcomes’ related to the abilities of diverse groups of adolescent girls and boys to help prevent, reduce and cope with conflict and promote peace in capacities related to their psychosocial wellbeing, acquisition of life skills and capacities for community engagement.In this regard, ADAP has developed the Adolescent Kit in harmony with a related initiative to develop a “peacebuilding competency outcome framework and toolkit.” Theoutcome framework of competency domains for ‘children as peacebuilders,’ is aligned with overarching goals in the areas of psychosocial wellbeing, life skills and participation for children and young people– with special attention to align outcomes with metrics for Life Skills Education in Formal and Non-Formal Contexts currently under development. That peacebuilding competency outcome framework and toolkit, scheduled for completion in September 2014, will offer relevant tools that will be adapted and used in the monitoring and evaluation process supported by this consultancy.

A team of ADAP consultants works together to develop the Adolescent Kit, which includes a technical lead coordinator, and a design coordinator with expertise in user-centred design. ADAP collaborates closely with colleagues, partners and adolescents at the field-level throughout the development of the Adolescent Kit. Through field visits and remote consultations inHaiti, Timor Leste, Kenya (with refugees from Somalia), Uganda, South Sudan, Jordan and IndonesiaADAP has gathered lessons and effective strategies to incorporate into the kit, and field tested activities guidance with facilitators and adolescents.The Child Protection, Education, C4D, Gender and Disabilities Sections, Innovation Unit, Supply Division, and Emergency Operations Division at headquarters also provide support and technical input.

The Adolescent Kit is currently under development. Sixty prototypes of the kit, each sufficient to reach 50 adolescents, will be produced inmid-September 2014. ADAP will collaborate with UNICEF Country Offices, partners and adolescents in Indonesia, Jordan, and one other country (TBD) to further develop and pilot test the kits in those countries, with the activities and deliverables of this consultancy contributing to that process. Findings from the collaborative development and testing process will feed into the final design of the Adolescent Kits, which will be produced at scale and made available for use globally in July 2015.

1. Purpose and expected results

From September 2014 to May 2015, ADAP will collaborate with UNICEF Country Office colleagues and partners in Indonesia, Jordan, and one other country (TBC)to develop and test the 50 prototype Adolescent Kits through new and ongoing programmes.The collaborations will take place in different programming contexts, and build on earlier consultations and other activities that took place in each of those countries between July 2013 and July 2014.

Currently ADAP is working with those country officesto finalize inception planswhich will specify a programme entry point through which the adolescent kit will be used as a resource, and outline country offices’ objectives in testing and contributing to the development of the kit. These inception plans will serve as the foundation for country-specific work plans for the collaboration and testing process. Those work plans will include specific details of the timelines and activities for ADAP, the UNICEF Country Office programme staff, and partners in each of the three countries.

A likely sequence of in each country is as follows:

1)Collaboration plans finalizedby 15 September 2015. ADAP team and Country Office teams to work together remotely.

2)In-country inception workshop.Two weeks, facilitated by the ADAP team with support from the UNICEF Country Office team and partners. The first inception workshop is tentatively scheduled for Jordan late September 2014.

3)In-country use, testing and development the Adolescent Kit.Four to six months, implemented directly by partners and adolescents, with support from the Country Team and remote support from the ADAP team.

4)Culmination/transition workshop.Two weeks, facilitated by the ADAP team with support from the UNICEF Country Office team and partners. All culmination/transition workshops completed by 15 May 2014.

The collaborative development and testing process in each of the three countries will be designed to address four major goals:

a)Testing the kit’s effectiveness toward outcomes for adolescents: to gather evidence as to whether using the kit approaches, activities and resources offered in the kit:

1. Result in reaching more of the hardest-to-reach adolescents.
2. Contribute to positive outcomes with respect to adolescents’ psychosocial wellbeing, learning of life skills, and positive engagement.

b)Testing the kit’s usability by UNICEF, partners and adolescents: Specifically, to learn:

1. Whether UNICEF programme staff at the country and field levels, partners and adolescents find the kit usable, especially with respect to the design and packaging of guidance, and the user-friendly digital and supplies.
2. Whether UNICEF programme staff and partners find it both adaptable and relevant to different programmes and contexts.
3. Whether adolescents find the activities relevant, engaging and worthwhile.
4. Whether the kit supports sustainable activities and processes for adolescents which they, facilitators and programme coordinators can use for the duration of a programme, and whichcan be adapted and/or institutionalized as the programming context evolves.

c)Enhancing and adding to the kit by gathering and incorporating new resources, strategies and lessons created at the country level as the kit is used: As UNICEF, partners and adolescents use the kit they will adapt it to their priorities and context, and will develop new approaches and activities. Through pilot testing and collaborative development process, those innovations will be documented for incorporation into future versions of the Adolescent Kit to enhance it and support cross-country learning.

d)To support capacity development of UNICEF offices and colleagues to implement approaches and activities for adolescents in the humanitarian action cycle and other low-resource contexts.

Built around those four core goals, the specific sub-objectives and plans for each of the three countries will be developed and adapted based on their respective programme objectives.

This entire collaborative development and testing process will result in evidence, lessons learned, and new ideas which will be applied to develop a final version of the Adolescent Kit, improving, refining and expanding on the prototype. In August 2015 the final version of the Adolescent Kit will be made available to all UNICEF Country Offices through the supply catalogue and used globally as part of UNICEF’s emergency response in the Humanitarian Action Cycle as well as regular programming. ADAP and other technical sections will provide ongoing training and support to country offices to use the kit through their programmes.

Tasks and responsibilities of the M&E Consultant

To support a strong, effective testing process, UNICEF will engage a consultant to coordinate the evaluation component of this collaborative development and testing of the prototype Adolescent Kits. The Evaluation [Coordination] Consultant will work closely with the two other ADAP consultants who are coordinating the development of the Adolescent Kit (a technical lead coordinator and a design coordinator, respectively). He or she will design and support a testing process that results in valid evidence, strong documentation and useful lessons to improve, refine and expand the next version of the Adolescent Kits.

The Evaluation Consultant’s work will focus on the two testing objectives outlined above (i.e. those having to do with the kits’ effectiveness and usability). His or her responsibilities will include:

• Developing and outlining country-specific testing objectivesfor each of the three countries, in line with the objectives identified in each inception plan.
• Developing plans for the testing process, in line with collaborating country offices’ priorities and availability, resources available, preliminary plans for collaboration that have already been established with each country, and other key activities that are part of the collaboration (i.e. those related to documentation and capacity development).
• Developingtesting instruments for the collaborative development and testing processes in each of the three countries.
• Facilitating and gathering evidence from the testing processes. This will likely include in-country testing processes such as focus group discussions and interviews, and remote processes such as discussions through online platforms and surveys. It will also include developing baselines, gathering feedback during the interim of testing, and gathering data at the culmination of activities.
• Preparing reports of findings from the testing experience, including actionable recommendations for how the prototype Adolescent Kit should be improved, refined, or expanded when the next versions of the Adolescent Kit are available and produced at scale beginning in July 2015.

In designing testing objectives and instruments to measure the kit’s effectiveness as a resource for positive outcomes with respect to adolescents’ psychosocial wellbeing, learning of life skills, and positive engagement consultant will use the Peacebuilding Competency Framework and Toolkit described above. As such, the process of developing and using instruments to test the prototype Adolescent Kits in this phase will also serve as a “usability” test of that framework and toolkit as a resource for peacebuilding oriented programmes. Depending on the findings from that aspect of the experience, testing plans and instruments may be preserved as sample materials for future use with both kits.

1. Expected Results

Through these contributions of the Evaluation Consultant, the following results will be achieved:

• Qualitative and quantitative evidence of the Adolescent Kit’s effectiveness as a programme resource will be gathered and documented. Evidence of gaps in the kit’s effectiveness will be articulated into specific recommendations to change, refine, add to and improve upon the prototype Adolescent Kit, recommendations which will be addressed in the design of a new, final version of the kit.

• Evidence of the Adolescent Kit’s usability as a programme resource will be gathered and documented. Evidence of gaps and challenges in using the kit will be articulated into specific recommendations to change, refine, add to and improve upon the prototype Adolescent Kit, recommendations which will be incoroprated into future versions of the kit.

• New activities, approaches and resources that are developed at the field level through use of the kit will be documented and incorporated.

• Test plans and instruments developed and used in the evaluation process will be preserved as programming tools for future programming with adolescents (in the context of the Adolescent Kit and beyond).

Evidence gathered during the testing process of the kit’s effectiveness and usability will serve to validate and foster confidence among new country programmes considering using it as a resource for relevant programmes. Lessons learned regarding gaps or challenges in using the kit will be used to make changes or additions as necessary. New activities, approaches or other concepts developed in the field will be documented and incorporated.

1. Deliverables

Expected Deliverables
1)Annotated outline of existing assessment tools which may be adapted for use in the pilot test evaluation process, based on desk research.
2)Outline of plans for testing process in each country, including indicators and assessment tools, based on the desk research report and a review of inception plans from each country office.
30 September
Indicators and assessment tools for use in Jordan. Draft report of findings from baseline/inception testing in Jordan. (Evaluation Consultant would travel to the country with the rest of the Adolescent Kit consultant team, and carry out these processes as part of the other inception activities). 15 October 2014
Indicators and assessment tools for use in Indonesia. Draft report of findings from baseline/inception testing in Jordan. (Evaluation Consultant would travel to the country with the rest of the Adolescent Kit consultant team, and carry out these processes as part of the other inception activities). 31 October 2014
Indicators and assessment tools for use in Indonesia. Draft report of findings from baseline/inception testing in third country TBC. (Evaluation Consultant would travel to the country with the rest of the Adolescent Kit consultant team, and carry out these processes as part of the other inception activities). 30 November 2014
Monthly interim reports of in-country testing activities and findings, developed through remote communications with country-office focal points and using any electronic tools and platforms.31 December 2014
Monthly interim reports of in-country testing activities and findings, developed through remote communications with country-office focal points and using any electronic tools and platforms.31 January 2015
Monthly interim reports of in-country testing activities and findings, developed through remote communications with country-office focal points and using any electronic tools and platforms.28February 2014
Final evaluation report from collaborative development and testing in Jordan, including documentation of useful lessons learned, test finding and annex with test instruments used. 15 March 2015
Final evaluation report from collaborative development and testing in Indonesia, including documentation of useful lessons learned, test finding and annex with test instruments used. 15 April 2015
Final evaluation report from collaborative development and testing in third country, including documentation of useful lessons learned, test finding and annex with test instruments used. 30 May 2015

1. Key competences, technical background, and experience required

-Demonstrated experience monitoring and/or evaluating child protection and/or education programmesfor children and young people in humanitarian assistance contexts, including: