Plan International

Posted: April 5, 2024

Deadline: April 12, 2024

Dili

M&E GEDSI Consultant

End Line Consultancy - SRHR

Plan International is one of the world’s largest INGOs, working with children and poor communities in 75 developing countries to advance Children’s Rights and Equality for Girls. We have worked with children, young people, and communities for 80 years, committed to making a lasting impact on the lives of the most vulnerable and excluded children while supporting children’s rights and gender equality. We support children’s rights from birth until they reach adulthood and enable children to prepare for and respond to crises and adversity. We drive changes in practice and policy at local, national, and global levels using our reach, experience, and knowledge. Plan International has been working in Timor-Leste since 2001 to promote the rights of all children and young people and give them practical support to achieve their rights to early education, water, sanitation and hygiene, employment and child protection, participation in disaster risk management, and promote gender equality. The plan works in two municipalities, Aileu and Ainaro. Please visit www.plan-international.org for more information.

Background/Context

Plan International Position Statement on SRHR

Plan International believes that all children, adolescents and young people have the right to make their own free and informed choices and to have control over their sexual and reproductive health and lives, free from coercion, violence, discrimination and abuse. Girls and young women[1] in all their diversity in particular are denied the ability to exercise these rights. Fulfilling the rights of all children, adolescents and young people is fundamental to achieving gender equality

To achieve this, Plan International implements three main strategies globally:

Support positive sexuality education and dialogue

Plan supports comprehensive sexuality education programs starting from early childhood through to young adulthood. CSE enables children and youth to explore values and attitudes and to build skills, regarding sex and sexuality. Plan delivers CSE in communities and enables CSE in schools; working with parents, community leaders, youth workers, teachers, civil society organizations, and government partners. Plan supports the integration of sexuality education in humanitarian settings, for example through the delivery of sessions in safe spaces.

Strengthen quality SRHR services

Plan works with public health facilities to enable more youth-friendly, gender-responsive, and inclusive sexual and reproductive health and rights services. Programs in Timor-Leste support Adolescents and youth access to information and services through the establishment of AYFHS ( Adolescents Youth Friendly Health Services) .

Support girls and young women most at risk

Girls and young women are disproportionately affected by and vulnerable to SRHR violations due to entrenched gender inequalities and the lower value attributed to girls and women in many societies. This is a heterogeneous group requiring targeted responses. Plan’s programs tackle stigma and discrimination, provide safe spaces and social networks, psycho-social support, and enable multi-sectoral services. Plan takes a multi-level, holistic approach to identify and address the root causes or ‘drivers’ of harmful practices including child, early, and forced marriage, and female genital mutilation/cutting, and works with relevant agents of change at all levels.

Therefore, and in alignment with these strategies, the project contributes to this vision by strengthening the capacity of local CSOs so that they can play a more dynamic role in promoting relevant activities to ensure that children, especially girls, enjoy their rights to SRHR and participation as well as to create an enabling environment for CSOs to act and to promote duty bearers to be accountable for the services provided to children.

The overall impact of the project is: “a more dynamic, gender transformative, and inclusive civil society that promotes the advancement of children's rights to SRHR and participation and equality for girls”. The main goal of the project is to reduce levels of early marriage, delay first pregnancy, and promote the active participation of girls, young women, and women in governance at the community, municipality, and national levels through participation in decision-making.

Project Intended Results:

The project has the following four expected outcomes:

Outcome 1: Children and Youth, particularly girls and young women, have improved knowledge on SRHR and are confident in making healthy decision on their bodies and SRHR

Outcome 2: Children and youth, in particular girls and young women, have strengthened knowledge on their right to participation and are able to influence community decision-making processes

Outcome 3: CSOs, including child and youth-led networks, have improved organizational and networking capacity and influence decision-making processes

Outcome 4: Duty bearers, including local government institutions and parents, have strengthened their understanding on their responsibility for children's rights and take action to support children, adolescents, and youth's right to SRH

The four-outcome put a slightly different emphasis on various aspects of the program, for instance in the outcome one makes a significant investment on improvement knowledge on SRHR and build the confidence of young people particularly girls to make healthy decisions on their bodies. In second outcome the program, focuses on strengthened knowledge on the right to participation and how to influence community decision-making process. The outcome three focus to strengthened capacity of CSO has and influence decision-making process and outcome four strengthening understanding on their responsibility for children rights and take action to support children, adolescents and youth right to SRHR. These projects under the programmed are encouraged to use Plan International Champions of Change methodology.

Project Core Activities:

Activities under Outcome 1

Output 1.1: Champion of change groups on SRHR, CP, and Gender Equality have strengthened the knowledge of children and young women on SRHR

A1.1.1 Young men participate in network meetings on SRHR issues

A.1.1.2 SRHR training to young men group for Champion of Change, including TOT

A.1.1.3 Gender Equality and Inclusion training for children and young people

A.1.1.4 Child protection training for children, young people, community members, parents, and teachers

A.1.1.5 Training for girls, boys, young men, and women trained on sexual reproductive health and rights (SRHR) at the community level, sub-district, and municipality level

A.1.1.6 Children and youth voice SRHR issues through community radio

A.1.1.7 Young men lead campaigns and conferences on SRHR issues

A.1.1.8 Develop/revise SRHR Guidelines/modules

Output 1.2: Young people's participation in comprehensive sexuality education (CSE) and access to SRH information have improved

A1.2.1 Training for young people in CSE program that addresses gender inequality and power in relation to reduce pregnancy, childbearing &STIs

A.1.2.2. Training for girls, boys, young women, and men in life skills, including reflection about the importance of respecting one’s body and other people’s bodies.

A.1.2.3 Training for young people on sex and gender, "Understanding gender-based violence as any form of violence against women and girls or individuals who did not conform to dominant gender norms

A.1.2.4 Support young people to access quality information and services on SRHR including STIs/HIV/AIDS, family planning, child health, and maternal health

A.1.2.5 Regular monitoring meetings to update progress, achievement, challenges, and solution of project implementation

Output 1.3: Improved implementation of SRHR curriculum and policies

A.1.3.1 Work with MOH and teachers to improve the implementation of SRHR curriculum and policies

A.1.3.2 Teachers Association's and PTA's regular meetings to discuss SRHR curriculum

A.1.3.3 Plan and partners support youth- led advocacy for the inclusion of SRHR curriculum in schools

A.1.3.4 Mid-term Review

A.1.3.5 Annual evaluation meeting

A.1.3.6 Exposure Study Visit

Activities under Outcome 2

Output 2.1: Women and girls are knowledgeable of their right to participate in decision making processes and express confidence in exercising that right

A.2.1.1 Socialization on CRC, CEDAW, DUDU, KNDL, CP, VBJ.

A.2.1.2 Capacity building on conflict resolution, leadership management, effective communication, and public speaking

A. 2.1.3 FGD with the young women representatives in suku council to increase their intervention in suku meeting

A.2.1.4 Drama/theater link to women's and girls' participation

A.2.1.5 Promote civic engagement and participation through workshops on women's and girls' participation

A.2.1.6 Support the existing 100% "Hau prontu" (I'm Ready) committee

Output 2.2 . Suku council members and men of the community have improved knowledge on gender equality and have the attitude to promote the participation of girls and women

A.2.2.1 Annual meeting with local authority to change perceptions and stereotypes about gender roles

A.2.2.2 Work in partnerships with men’s organizations and traditional leaders to reinforce the changes in societal attitudes towards adolescent girls and women

A.2.2.3 Men and young men lead the celebration of International Women's Day, Girls' Day, and International Children's Day

A.2.2.4 TV and Radio Broadcasting related to Women's and Girls' Participation topic

Output 2.3 Women's participation in household income generating activities have improved

A.2.3.1 Establish and support VSLA and Horticulture groups

A.2.3.2 Training on savings and loans, horticulture, business skill and finance, cost analysis and market linkages

Activities Under Outcome 3:

Output: 3.1 CSOs’, youth-led CBOs', and Child Right Coalition's capacity are strengthened to prevent and respond to SRHR issues and promote women's and girls' participation

A 3.1.1. Trainings on organizational management, financial management system, human resources management, program approach, technical guidelines, result monitoring and reporting

A.3.1.2 Technical trainings on SRHR and Participation

A 3.1.3 Reactivate and strengthen Child Right Coalition (CRC) to implement joint advocacy action for child rights, with a focus on SRHR and participation

Output 3.2 CSOs, youth-led CBOs, and Child Right Coalition support children and youth, particularly girls and young women, to actively participate in voicing their concerns and needs on SRHR and participation

A 3.1.1. Trainings on organizational management, financial management system, human resources management, program approach, technical guidelines, result monitoring and reporting

A.3.1.2 Technical trainings on SRHR and Participation

A 3.1.3 Reactivate and strengthen Child Right Coalition (CRC) to implement joint advocacy action for child rights, with a focus on SRHR and participation

A.3.2.1 Children and youth participate in the regular meeting on SRHR and gender networking meeting

A.3.2.2 Children and youth attend municipality level workshop/seminar on participation and SRHR

A.3.2.3 Regular CSOs meetings at the municipality level and develop the progress activity report to share with the community and government

A.3.2.4 Support the Child Rights Coalition to implement influencing activities to improve and establish laws and policies that will help children and young people in the realization of their rights, with a focus on SRHR and participation

A.3.2.5 Organizational Development for CSO Partners

Output 3.3: CSOs, youth-led CBOs, and Child Rights Coalition have strengthened knowledge on policy development and budget allocation process

A.3.3.1 Trainings for CSOs, Youth-led CBOs, and Child Right Coalition on government state budget, social audit, and good governance, monitoring, and advocacy

A.3.3.2 CSOs, Youth-led CBOs, and Child Right Coalition are supported to conduct monitoring to government project implementation and expenditure and present the results at relevant workshops

Activities Under Outcome 4:

Output 4.1 Relevant government institutions have the expertise and resources to promote children and youth SRHR and participation

A,4.1.1 Civic education on SRHR and women's participation for parents (male & female), children (girls & boys), and youth (male & female)

A.4.1.2 National and municipality awareness raising (workshop, dialogue) on SRHR and women's and girls' participation

A.4.1.3 Support government institutions in consulting with CSOs on resource allocation and training on good governance and state budget

Output 4.2 Children and youth, particularly girls and young women, actively participate in voicing their concerns and needs on SRHR and participation

A.4.2.1 Advocacy campaigns, social trainings and communication skills for children (girls and boys)

A.4.2.2 Organize girls conference on different topics related to SRHR and the participation of girls and women

A.4.2.3 Support girls to lead long march, campaigns, lead IDG events, lead International Children Day and Women’s' Day

Output 4.3 Community and religious leaders have the knowledge and skills to support the rights of children to SRH and participation, in particular girls and young women

A.4.3.1 Community male dialogues on SRHR, the right of participation, gender equality and children's protection

A.4.3.2 Socialize customary laws (tarabandu) and other legal policies and laws related to SRHR and women's participation

A.4.3.3 Drama and theater on child rights, SRHR, and women's and girls' participation.

Output 4.4 Parents, teachers, and the communities have an improved understanding and attitude on SRHR

A.4.4.1 Training for parents, teachers, community members on SRHR to support their children and young people in their development process

Output 4.5 Health facilities take action to provide quality adolescent-friendly, gender-responsive and inclusive SRHR services

A.4.5.1 Operational costs, logistics, IEC materials, Peer counsellors’ incentives, travel allowance

A.4.5.2 Training of staff in targeted health facilities, and social workers, on National Guidelines for YFHS

A.4.5.3 Two launch events to promote the AYFSH in Aileu and Ainaro

A.4.5.4.Expansion of AYFHS to other clinics and areas

Project Duration and Targeted Locations:

The project is implemented during the period of 1st of June 2019 till 30 June 2024. The project is implemented in two target locations / two municipalities of Aileu and Ainaro, and targeted 51 community/villages ( 21 in Ainaro municipalities and 30 in Aileu municipalities), for more detailed information about the project please see the table below

Project Name

For a Dynamic and Gender Transformative Civil Society to promote SRHR and the Participation of Girls and Women in Timor-Leste 2019 – 2024.

Location

Aileu and Ainaro Municipality

Project Period

1st July 2019 – 30th June 2024

Targets Groups & Final Beneficiaries

CAY (Children, Adolescents, and Youth)
Parents
Teachers
Community
Local leaders, traditional leaders, and faith leaders
Duty bearers

Key stakeholder groups

Government Institution: MOH, MOE, SEI, MSSI, INDICA
The Ombudsman for Human Rights and Justice (Provedoria dos Direitos Humanos e Justiça- PDHJ)
Civil Society Organization such as (Forum ONG Timor-Leste-FONGTIL), Concelho Nacional da Juventude de Timor Leste (CNJTL),
Media such as national TV and community radio stations in 2 municipalities.
Youth with all diversity
Faith Leaders
Youth-led CBOs
Youth Volunteer
Youth Activism

Implementing partners

FADA Association in Aileu Municipality
TALATA in Aileu Municipality
NAFOFILA in Ainaro Municipality

The overall Purposes of the End-line Evaluation

The end-line evaluation will be contributed to the global final evaluation, by providing data and insights that can be used for analysis. Specifically, the evaluation will contribute to the analysis of the three global outcomes identified for the project.

The 3 global indicators :

Individuals and groups have strengthened knowledge, attitudes, and behaviors on children's and young people’s rights and gender equality
Civil society, particularly child- and youth-led collective actions, have strengthened legitimacy and diversity, enhanced space and influence, and improved collaboration within civil society and with duty-bearer
Duty bearers have strengthened policy, systems, and services affecting children's rights in response to the engagement and advocacy of civic actors

By assessing the impact, effectiveness, and challenges of CSS interventions, the evaluation seeks to identify lessons learned and best practices that can inform future programming and policy development in the field of sexual and reproductive health.

Description & Objectives

Purpose:

The main purpose of the end-line evaluation is to measure the overall impact of the “ For a Dynamic and Gender Transformative Civil Society to promote SRHR and the Participation of Girls and Women in Timor-Leste” project. It is to assess and verify the outcomes/results achieved compared to the set objectives and how they have contributed to the fulfillment of the project goal and to examine the effectiveness and sustainability of these achievements. Besides assessing the outcomes/results achieved, the end-line evaluation also helps identify lessons learned from the implementation of the “For a Dynamic and Gender Transformative Civil Society to promote SRHR and the Participation of Girls and Women in Timor-Leste” project. This includes understanding what worked well, what challenges were faced, and what strategies were most effective in achieving the project's goals. These lessons can inform future programming and contribute to knowledge sharing within the field of SRHR. It is also to facilitate a process that will document project outputs and outcomes and the project’s contribution to impact considering the baseline study and MTE that was conducted at the beginning and middle of the project. Eventually, the process should mobilize the various stakeholders to take action based on this documentation.

Objectives:

Evaluate the project's contribution to achieving the stated goal, outcome, and outputs outlined in the results framework.
Assess the performance and achievements of the project's interventions in establishing sexual and reproductive health.
Examine the intended and unintended effects of the project, reviewing the approaches and strategies employed, and assessing the impact on the overall condition and position of beneficiaries, particularly women and girls.
To contribute to the generation of knowledge and evidence in the field of SRHR, by sharing findings and lessons learned from the evaluation with the broader public health community.
To document best practices and innovative approaches that have emerged during the implementation of the “ For a Dynamic and Gender Transformative Civil Society to promote SRHR and the Participation of Girls and Women in Timor-Leste” project, with a view to replicating successful strategies in future programming.
To document best practices and innovative approaches that have emerged during the implementation of the For a Dynamic and Gender Transformative Civil Society to promote SRHR and the Participation of Girls and Women in Timor-Leste” project, with a view to replicating successful strategies in future programming.
To provide evidence-based findings and recommendations that can inform decision-making by project stakeholders, including donors, policymakers, implementing agencies, and communities affected by the project
Develop specific recommendations to guide future program management and design, considering the project's multiple stages and aiming for a comprehensive vision of the project phases.
Evaluation and assessment of how the gender was mainstreamed throughout the project cycle. This will guide our gender analysis in the final report.

The key questions to be answered by the evaluator:

Relevance:

To what extent were the project's goals and outcomes aligned with the needs and priorities of the target beneficiaries, particularly girls and young women?
How well did the project address the SRHR topics?
Were the project interventions and activities responsive to the cultural context and challenges?
How does the SRHR project address issues of equity and inclusivity? This includes considering whether the project reaches and benefits marginalized and vulnerable populations, such as adolescents, women and girls, sexual minorities or LGBTIQ+, and people living with disabilities
Is the SRHR project aligned with national policies, strategies, and priorities related to sexual and reproductive health? This involves examining whether the project supports and complements existing government initiatives and frameworks and whether it contributes to achieving broader national health and development goals.

Effectiveness:

To what extent did the project interventions achieve the desired outcomes and outputs as outlined in the results framework?
To what extent has the SRHR project contributed to positive changes in knowledge, attitudes, and behaviors related to sexual and reproductive health? This involves assessing whether project interventions have successfully influenced behaviors such as accessing reproductive health services and information
To what extent has the SRHR project built the capacity of CAY, parents, and communities to address sexual and reproductive health issues? This involves assessing whether the project has strengthened the knowledge, skills, and agency of target beneficiaries to make informed decisions about their sexual and reproductive health, advocate for their rights, and engage in community-based initiatives.
How well the Civil Society Strengthening programme approach (composed of (1) civic space and influence (2) collaboration and (3) legitimacy and diversity) has been applied into the project planning, implementation, and monitoring process?
How successful were the efforts to establish positive protection mechanisms, change gendered beliefs and social norms, and enhance knowledge and skills on gender equality and sexual and reproductive health?
What evidence is there of the project's impact in reducing gender-based violence, harmful traditional practices, and socio-economic difficulties?
How well has the internal monitoring system contributed to the evaluation and to ongoing project implementation?

Efficiency:

How efficiently were project resources, including financial, human, and material resources, utilized throughout the project implementation? This involves examining the allocation of resources to different project activities and interventions, as well as assessing whether resources were used effectively to achieve desired outcomes.
How efficiently were the project resources allocated and utilized to achieve the desired results?
Did the project adopt cost-effective strategies in delivering the interventions?
Were there any challenges or bottlenecks that hindered the efficient implementation of the project?
How effectively did the project leverage partnerships and collaborations to enhance efficiency? This includes evaluating the extent to which the project engaged with relevant stakeholders, including government agencies, civil society organizations, and community-based groups, to leverage resources, expertise, and networks to achieve project objectives.
Were monitoring and evaluation systems effectively implemented to track project progress, identify challenges, and inform decision-making?

Impact:

What has been the progress made towards the achievement of expected activities and results of all project components?
What unintended consequences, both positive and negative, have resulted from the SRHR project?
How has the project contributed to changing social norms, beliefs, and attitudes towards SRHR and harmful traditional practices?
To what extent the SRHR project improve access to sexual and reproductive health services and information ,particularly among marginalized and vulnerable CAY?
What have been the changes in duty bearers’ knowledge, skills, attitudes, and behaviour as a result of their interaction with this project?

Coherence:

To what extent were the project interventions and activities aligned with the overall project goal, outcomes, and outputs as stated in the results framework?
Was there a coherent and coordinated approach in implementing the project activities, involving key stakeholders, partners, and beneficiaries?
Does the SRHR project take a comprehensive approach to addressing sexual and reproductive health issues?
Is there coherence in the approach and strategies used to implement the SRHR project?
Does the SRHR project take a comprehensive approach to addressing sexual and reproductive health issues?
Is the SRHR project harmonized with national policies, strategies, and priorities related to sexual and reproductive health?

Learning

To what extent have the target beneficiaries increased their knowledge and understanding of sexual and reproductive health issues as a result of the project?
How do we learn from the project and share the learning to improve programmes?
Has there been a culture of feedback and reflection within the SRHR project, allowing for continuous learning and improvement?
What needs to be done differently to achieve and maximise a positive impact on children’s lives and the fulfilment of their rights?

Sustainability:

To what extent are the project's outcomes and positive changes likely to be sustained beyond the project duration? Are their evidence of that the benefits of the project will last after the project is finished
To what extent do communities demonstrate ownership of the SRHR project and its outcomes?
How well has the project engaged and mobilized the community, key stakeholders, and local partners to promote sustainability?
How well have project interventions been integrated into existing health systems and structures at the local, regional, or national level?
Are there robust monitoring and evaluation systems in place to track progress, identify challenges, and inform decision-making for sustaining SRHR interventions?

Civil Society Strengthening

How the CSO (NGO partners) have been applied with their Organizational Development (OD) plan and what areas to be improved?
In what ways are CSOs changing/improving because of organizational support received through the Generation Change Programme?
To what extent and how have the Generation Change improved civic space, including managing related risks, and contributed to enabling an environment for civil society, especially children and youth-led collective actions?
What are some of the best practices, innovations, or advocacy solutions developed in the Generation Change Programme contributing to the realization of its overarching goal/s? Which of these could be scaled up?
Has the program contributed to new types, more/better collaboration within civil society, particularly for children or youth-led CSOs or those representing marginalized groups? If so how?
In what way, did partnering with Plan International help CSO partners expand, connect, and collaborate with other coalitions and/or networks beyond the project issues?
Has the program contributed to improved or expanded space and opportunities for the CSOs, particularly child- and youth-led, to influence and/or engage with duty bearers at national and sub-national level? Consider any negative effects of COVID- and shrinking space on this

Child rights, gender and inclusion:

To what extent has the project applied gender transformative approaches and explicitly contributed to gender equality results that improve the rights of girls, boys, and young people in all their diversity?
Questions based on the targeted elements of gender transformative change:
Has the programme been coherent in terms of the Plan’s gender transformative approaches (six elements of gender transformative approach) applied and what can be strengthened moving forward?
What has been changed in terms of gender transformative approach (focused on six elements of gender transformative approach)?
What are the root causes of gender inequality/ gender and social norms related to the programme’s thematic focus that have been tackled? (Element 1)
How has the programme improved girls'/ young women's agency (knowledge, confidence, ability, empowering, and lead action) to challenge and influence others for gender equality and inclusion? (Element 2)
How has the programme worked with men and boys to change the patriarchal norms and promote positive masculinities toward gender equality? (Element 3)
How has the programme improved equality conditions (welling, equal resources, and opportunities) and position (social stature, equal value, respect, and access to power of decision making) of girls/ young women in all their diversity? (Element 4)
How has the programme worked with diversity and taken an intersectional lens to engaging with girls, women, boys, and men in all their diversity for change in particular, ethnic minorities, indigenous groups, and people with disabilities? (Element 5)
How has the programme created a more “enabling environment” (Element 6) and strengthened societal structures, systems, engaged duty-bearers, and NGO partners to enable gender equality across the different stakeholder groups of individuals, communities, civil society, service providers, and policymakers? What were the policies and services improved to enable gender equality and inclusion?

Users

The intended users of the evaluation will be those directly involved in and/or managing the project. The evaluation will help Plan Timor-Leste and the project partners to learn from the project implementation to guide future programming, project design, and advocacy efforts. The evaluation report will also be used to document the progress of the project to the donor and to extract key achievements for communication purposes. The end-line evaluation report also will be used by CSOs who working on sexual and reproductive health advocacy, rights-based approaches, and community mobilization. They can use the findings to advocate for policy changes, raise awareness about SRHR issues, and strengthen community engagement and empowerment.

Overall, the end-line evaluation report of an SRHR project is a valuable resource for a wide range of stakeholders involved in promoting sexual and reproductive health and rights. It serves as a tool for accountability, learning, advocacy, and evidence-based decision-making to improve the health and well-being of individuals and communities.

All materials, tools, and other documents used/produced under this task will be shared with Plan International Timor-Leste, and their copyrights belong to Plan International Timor-Leste.

Geographical scope:

The End-line evaluation will be conducted at the local and national levels. At the local level community, the research activities will cover two municipalities of Aileu and Ainaro and 51 targeted communities. At the national level, the research activities will involve other participants from CSOs, Government institutions such as the Minister of Health (MOH) and Minister of Education (MOE), and other relevant stakeholders.

The geographical scope of the end-line evaluation report provides context for understanding the reach and impact of the SRHR project, as well as the diversity of settings and populations served. It helps stakeholders to assess the relevance, effectiveness, and sustainability of project interventions within different geographic contexts and to identify lessons learned for future programming and policy development.

Methods for data collection and analysis:

This End-line evaluation employs a participatory approach to ensure that children, young people, and civil society partners will be at the centre of the final evaluation and that they are provided with the opportunity to reflect on achievements, challenges, and lessons learnt. The evaluation also employs a mixed methods approach to ensure that triangulation of information gathered is possible. In alignment will the tools developed at the baseline stage and the M&E plan requirements, the evaluator(s) will review/develop appropriate quantitative and qualitative tools for data collection with different stakeholders adopting child-friendly and age-appropriate approaches. For the quantitative approach, Plan Timor-Leste recommends using one of the 3 different approaches: randomized controlled trial, comparison group, or pre-post comparison.

At the beginning of the process, the evaluator and relevant Plan staff will conduct meeting(s) to further discuss and agree on the methodology and approach proposed by the evaluator in his/her proposal; including:

The overall strategy, approach, samples, locations, logistics, coordination, and implementation plan of the study.
Review of the tools proposed as well to ensure our gender transformation targeting and safeguarding either in tools and methodology but also to train the data collectors.
Identify the key internal and external stakeholders to be involved at different stages including the process of youth and community participation; and opportunities to discuss and respond to emerging findings as relevant.
A time plan including milestones and management reporting from a consultant.
Use and dissemination plan for the study including reporting formats or events, audiences, and main responsibilities.
How data will be handed over to relevant Plan offices at the end of the process.
Safeguarding Children and Young People policy and ethical considerations during the implementation of the study and gender equality policy

Note: it is expected that the consultant will develop a detailed methodology for data collection, data management, and analysis in their proposal. This methodology should be in direct response to the Evaluation questions.

Secondary data review (desk review):

The consultant with support from Plan International Timor-Leste will identify secondary data for review (research, evaluations, reports, and so on relevant to the project). These will include but are not limited to:

Project proposal document
Project logical framework and Monitoring and Evaluation Plan
Project baseline study and MTE reports
Workshop meeting reports
Annual project reports
Success stories
Gender Marker
Risk Matrix
Plan International Safeguarding Children and Young People Policy and Code of Conduct
Child Rights Situation Analysis – Plan International and Save the Children

Primary data collection and data entry:

Following the desk review, the consultant will design the final evaluation tools to collect the primary data for the project. The study will include qualitative and quantitative approaches with a variety of primary and secondary data sources. The consultant will develop an electronic database to store relevant data from the final evaluation linked to all the indicators.

Plan always aims to integrate the elements of its Programme and Influence Quality principles in carrying out project final evaluations. Plan is also interested in seeing the final evaluation process as encouraging reflection and dialogue among all the main stakeholders involved in the social issues we work on, from their perspective as well as ours.

Tools:

The consultant is expected to draft and present the methodologies and appropriate tools for the intended evaluation and consider the following guidance notes:

Both qualitative and quantitative data should be collected and reported and data sources cited. It is also expected that studies, focus group discussions, Key informant interviews (in-depth interviews), and case studies will also be collected.
The methodology should discuss relevant sampling techniques, questionnaires applying the Washington Group Question, appropriate data collection tools and methodology, and any statistical package to be used for data analysis
The consultant will design and propose an appropriate methodology and tools to be agreed on with the Country Office and should be in line with the programme level on M&E system. Qualitative and quantitative data will be generated and analyzed concerning the project results framework. The consultant will also propose an appropriate sampling methodology and size that is statically valid and cost-effective. The sample size and proposed tools will be discussed and approved by Plan before the study.
When deciding on the composition of participants, special attention will be paid to the principles of inclusion and non-discrimination. Gender balance and the involvement of excluded and marginalized groups from communities in which the plan works must be ensured

Sampling:

The consultant will Pre-identify the outreach tools of the sample of the target group and will provide the suggested sampling methodology and size and vulnerability profile identification of the CAY ( Children, Adolescents, and youth), particularly young women and men. Plan will be reviewed and decide whether to approve it or if any further modifications or changes are needed.

The inquiry should be conducted by using key informant interviews (KII) and focus group discussions, using Vignettes where possible. Key informants are people with privileged positions in society who gain a special vantage point to look at the issue, such as by being engaged professionally, administratively, politically, and so on.

Disaggregation of data: All the collected data should be disaggregated by age, sex, location, disability, and sponsorship status.

It is important to note that the study will be conducted within normal monitoring, evaluation, research, and learning (MERL) activities within the project. All data collection should be done with targeted beneficiaries who were targeted by interventions and benefited from the programme’s services. Any data will be collected from Plan International projects’ beneficiaries will be anonymous and confidential according to PI general policies on data privacy and will not be shared with any third parties unless through general recommendations for improving the community’s status.

The sample size of the end-line evaluation will be based on a discussion with a consultant by using a random sample. Plan International will have a deep discussion with a consultant to decide the number sample size and municipalities

Analysis, Validation and reporting:

It is critical that findings are validated by a representation of all stakeholder groups who participated in the evaluation. For this reason, after data analysis and the first drafting of the evaluation report (also incorporating the desk review), the evaluator will conduct a validation for the project evaluation. The consultant will consider the validation activity in his/her proposal and financial offer. Plan International Timor-Leste and the consultant will agree on the exact scope, level, and time of this validation process while refining the detailed implementation plan. The feedback from the validation will be reflected in the draft report submitted to Plan for further feedback and comments to be considered in the final evaluation report.

Dissemination of findings:

Before finalization of the consultancy, the consultant will share the main findings with the project’s stakeholders and research participants in an audience-appropriate way.

Deliverables and Timeline

Key Deliverables

Deliverable

Format

Length

Due

Detail

Inception report

The inception report as well as all the suggested tools should be delivered in English and Tetun in a Word document.

Does not exceed 40 pages

Two weeks

after the initial meeting and desk review have taken place.

The report should include:

An updated timeline;
detailed methodology, including draft sampling methodology and size;
draft data collection tools; (e.g. Proposed guiding questions for semi-structured interviews, focus group discussion or other approaches, and proposing the draft data collection questionnaires)
ethical considerations;
consent forms for any primary data collection;
(draft) methods for data analysis;
brief justification of the methods and techniques used (including relevant underlying values and assumptions/ theories) with a justification of the selections made (e.g. of persons interviewed).
The key internal and external stakeholders to be involved
Summary of the preliminary findings and any preliminary hypotheses resulting from the desk review and explain the process for obtaining the participants’ consent.

Data collection tools and related materials

Word and excel, in English

One week after the inception report

The final FGD questions, interview questions, questionnaires, other tools that will be used with target groups will be shared by the consultant in English prior to the data collection process after the tools have been tested and possible modifications to them based on the tests have been made.

Draft assignment report:

The draft report should be delivered in a soft copy in English. References should be fully cited after all important facts and figures.

A full report with main text of 40 pages excluding cover page, basic document information page, table of contents, abbreviations, executive summary and annexes.

3 weeks after the inception report

The report should as a minimum include the following elements:

Front page with the title of the end-line evaluation , date and authors of the report
A table of basic document information on page 2
Executive summary (3-4 pages) that presents the key points of the different sections
Objectives and the intended use of the end-line evaluation
Methodology and limitations
Description of the project
Findings, including a table presenting the baseline data and suggested targets for each of the indicators in the project’s logical framework
Conclusions and recommendations
Relevant annexes, which as a minimum must include:
List of people interviewed or consulted
Terms of Reference for the endline evaluation

Final assignment report

In a soft copy in English

40 pages

One week after the draft report

The endline evaluation report will be considered final only after incorporating the feedback from stakeholders, Plan International Timor-Leste and partners (including the same components of the draft report)

Power point presentation

Power point in both Tetun and English languages

Power point presentation with the main findings of the evaluation , and presenting the results to project’s staff and partners.

Expected Deliverables/Output

The expected deliverables/output that needs to consider by the consultancy as below:

An inception report with detailed study methodology, sampling methodology, target respondents, including an outline of tools and processes to be used for conducting the End-line Evaluation and a feasible work plan.
Data collection tools and related materials;
Submission of clean datasets for both qualitative and quantitative data interviews
Draft Endline Evaluation report to be presented to Plan International (no more than Max 40 pages, including an executive summary not exceeding 3 pages).
A validation workshop focuses on the draft final report
A final report, max 40 pages, including cover sheet, executive summary, description of objective, methods, and limitations, a summary of data, finding, and conclusion plus appendices that include the TOR, data collection tools, results framework indicators with evaluation data and the proposed annual and cumulative target and direct and indirect beneficiary numbers.
All materials produced by the Evaluation including data excel, hard and soft copies of the report, and other related materials in the soft form will be submitted to the Plan International Timor-Leste for Data Privacy policy.

Ownership:

Raw data will be delivered to Plan International Timor-Leste at the end of the study:

All original study instruments with their recorded field data.
Copies of all Excel files / databases used for data analysis (English and Tetun versions, for Plan Timor-Leste to validate the analyses).
All materials, reports and other documents produced by the consultant under this task will be the sole property of Plan International Timor-Leste
For any use of the produced materials/reports/documents, other than for the purpose of the project and Plan International Timor-Leste, the consultant must receive a formal approval from Plan International in writing.

Permissions:

The consultant will be responsible for obtaining the necessary permissions to implement the research data collection at the local level.

Ethics and Child Protection

Plan International is committed to ensuring that the rights of those participating in data collection or analysis are respected and protected in accordance with Ethical Monitoring, Evaluation, Research, and Learning Framework and our Child and Youth Safeguarding Policy. All applicants should include details in their proposal on how they will ensure ethics and child protection in the data collection process and do not harm. Specifically, the consultant(s) shall explain how appropriate, safe, and non-discriminatory participation of all stakeholders will be ensured and how special attention will be paid to the needs of children and other vulnerable groups. The consultant(s) shall also explain how the confidentiality and anonymity of participants will be guaranteed during the process.

Ethical review:

An Ethical Review is conducted to review whether MERL initiatives are in line with Plan International’s MERL Policy, MERL Standards, the Global Policy on Safeguarding as well as the key ethics principles outlined in the Framework for Ethical MER. This prioritizes the principle of ‘not harm’ and ensures the well-being, dignity, rights, and safety of children, young people, and other participants in data collection are respected and protected.

Timeline

Task

Days of work

Responsible

Individuals Involved

Tendering / Interview Candidate

Plan

HR, Project team

Background Checks and Contracting;

Endline evaluation Consultant

Plan

HR & Security Focal point

Initial meeting:

Update from Project team and Desk review: review relevant project documents, project proposal, & relevant documents

Plan