Evaluation purpose This summative evaluation is expected to determine the extent to which the project has achieved the short medium and long term goals it set out to achieve, to document what worked and what did not work under what circumstances. The evaluation is expected to also carefully assess the extent to which the project logic/design and implementation approaches were adequate for the nature of the problem is was funded to address and the context in which it was implemented. Furthermore, the evaluation findings and recommendations are expected to inform the continuous improvement in the current and future similar initiatives by IOM Egypt and its government partners. The evaluation is also expected to document lessons and good practices that will specifically inform the improvement in the implementation of future counter trafficking interventions and design effective interventions in the future. Evaluation users include the direct project management team, IOM Egypt leaders who are interested to ensure the integration of lessons learnt in the future projects/programmes. The evaluation findings are also expected to inform donor’s decisions on whether to fund the follow up phases based on opportunities and gaps identified during the project implementation and evaluation.
Evaluation scope The evaluation will cover the whole project period 1 January 2018 – 30 September 2020 of the project “Strengthening the Sphere of Protection for Victims of Trafficking and Vulnerable Migrants in Egypt while Pretending Irregular Migration and Human Trafficking” (SPHERE)”. All project activities are included within the scope of the evaluation. Some project activities were conducted in governorates outside Cairo, such as Alexandria and the top six sending governorates for irregular migration – Assiut, Beheira, Gharbeya, Dakahleya, Fayoum, and Kafr El Sheikh – the need to conduct field visits will be decided with the evaluator based on the chosen data collection and evaluation tools.
Evaluation criteria The evaluation is expected to assess project performance against the revised OECD/DAC evaluation criteria: relevance, effectiveness, coherence, efficiency, impact and sustainability.
Evaluation questions Below are the key generic evaluation questions that the evaluation is expected to answer. The evaluator may develop additional specific evaluation questions as necessary in line with the
evaluation purposed in order to collect quality and sufficient data needed to meet expected standard of evaluation report.
i. Relevance – To what extent has the project responded to the needs of various priority beneficiary groups and stakeholders? – Has the project adequately responded to local and national priorities? – To what extent is the project consistent with IOM priorities and mandate?
ii. Effectiveness – To what extent was the project implementation fidelity been ensured? Were all project activities implemented as originally planned and in a timely manner? If not, why? – To what extent has the project activities effectively produced quality outputs and outcomes as planned? – Which of the project strategies were more effective and which were least effective in producing planned short and long-term results and why? – What could have been done differently (design and implementation approaches) to make sure that the project more effective in reaching short and long-term target results? – What (if any) lessons can be drawn from the project?
iii. Efficiency – To what extent were project implementation strategies the most cost-efficient? – Could activities have been successfully been implemented with fewer resources without compromising the quality of resultant outputs and outcomes? – Could cheaper alternative implementation strategies/packages have reached similar results or more? – How successful has the project been in synergizing and complementing interventions of other national and international actors and leveraging non-project resources?
iv. Impact – What impact (positive and/or negative, intended or unintended) did the project have on its’ beneficiaries and relevant stakeholders and what do these actors perceive to be the project’s impact? – What observed changes in attitudes, capacities and institutions etc. can be linked to the broader framework of the project’s interventions? – What project interventions strategies were effective in triggering the observed medium and long-term results (outcomes)?
v. Sustainability – What project activities, outputs, outcomes and benefits brought about by the projects are likely to live on or continue after the project has ended? – What actions are recommended on the project design and implementation to strengthen sustainability of future interventions? – What potential exists for the continuation, replication or scaling up of the project’s results by national partners?
vi. Cross-cutting issues – To what extent has the project addressed cross-cutting issues such as gender, human rights, and the environment?
Evaluation methodology The evaluator will be responsible for developing a detailed evaluation methodology that will be followed to respond to the above evaluation purpose and questions. However, it is expected that a mix methods approach will be followed to get diverse data required to reach an objective assessment of whether the project achieved what was set out to do and draw useful recommendations and conclusions. Therefore, it is expected that a combination of quantitative and qualitative methods of data collection will be used including document review, key informant interviews, surveys, and other data collection methods as necessary to satisfactorily respond to the above set of evaluation questions. IOM will share relevant background documents and project records as needed by the evaluator or evaluation team. These will include, amongst others, progress, training and other reports, institutional documents, meeting minutes as well as other relevant information products produced during the implementation. The evaluation must follow the IOM Data Protection Principles, UNEG norms and standards for evaluations, and relevant ethical guidelines.
Evaluation deliverables The evaluator is expected to submit the following deliverables:
- Inception report comprising detailed evaluation approach and methodology, a set of data collection instruments as well as a detailed work plan that will be developed in close consultation with IOM Egypt project management team in coordination with Regional M&E Officer.
- Draft analytical evaluation report which the evaluation must submit and present to the management team for inputs.
- The final analytical evaluation report that incorporates comments and feedback from IOM Egypt management team and Regional M&E Officer. The final report submitted to IOM must be in English and include (i) Executive summary; (ii) Project background; (iii) Evaluation background and methodologies; (iv) Project performance; (v) Achievements against expected outcomes/results, (vi) Sustainability of the achievements and overall strategy; (vii) Good practices and lessons learned of the project; (viii) Summary of conclusion and recommendations. The content of the report should be clear, and all evaluation conclusions must be substantiated and backed by evidence. The report should bring new perspectives to the subject evaluated and include the comparison of the baseline data collected at the outset of the project and impact/changes made after the implementation, and should not exceed 45 pages, excluding annexes. Where there is no baseline, the evaluation methods is expected to include retrospective questions to reconstruct the baseline condition as reference for measuring achievement.
Evaluation brief: the evaluation is also expected to submit a two-pager evaluation brief in English. The brief should provide a succinct summary of the evaluation, the key findings, lessons learnt and recommendations.
Evaluation workplan The evaluator will supply a first evaluation draft to the Project Manager no later than 15 November 2020 for comments, and following feedback, the final report should be submitted no later than 30 November 2020. The table below provides provisional timeline for key evaluative activities. The final detailed evaluation work plan will be agreed upon between IOM Egypt and the evaluator.