Evaluation Consultant - Evaluation of ICAO's 2026-2028 Business Plan

International Civil Aviation Organization; Montreal; Presencial; CAD 90.000 - 130.000; Jornada completa

Descripción de la vacante

International Civil Aviation Organization in Montreal seeks an expert evaluator to lead an independent mid-term evaluation of its 2026–2028 Business Plan. The consultant will assess relevance, coherence, efficiency and sustainability, aligning with ICAO’s strategic plan and governance framework.

The role requires a Masters or equivalent, at least ten years in evaluation, strong data and stakeholder engagement skills, and fluency in English for high-quality reporting under UNEG norms.

Formación

• A minimum of ten years of experience in evaluation and complex evaluations.
• Expertise in theory-based, utilization-focused and mixed-methods evaluation.
• Strong quantitative and qualitative analytical skills for governance and reporting.
• UN system or multilateral organization evaluation experience with UNEG knowledge.
• Ability to engage senior management and stakeholders with independence and confidentiality.
• Excellent English report-writing, facilitation and presentation skills.
• Ability to deliver high-quality, evidence-based products within deadlines.

Responsabilidades

• Prepare inception report, evaluability assessment and evaluation design.
• Review business and strategic documents; map indicators, data sources and risks.
• Synthesize evidence and prepare draft/final evaluation report with recommendations.
• Present findings, validate with stakeholders and ensure quality assurance.

Conocimientos

Evaluation

Theory-based evaluation

Utilization-focused

Mixed-methods analysis

Stakeholder engagement

Report writing

Educación

Master's degree or equivalent

Bachelor's degree + experience

Descripción del empleo

JOB DESCRIPTION

The Organizational Setting

The International Civil Aviation Organization (ICAO) is a specialized agency of the United Nations and the global forum through which its 193 Member States cooperate on international civil aviation. Guided by its vision of achieving the sustainable growth of the global civil aviation system, ICAO develops policies and international Standards, undertakes compliance audits, conducts studies and analyses, provides support, and strengthens aviation capacity in cooperation with Member States and other stakeholders. Since its establishment in 1944 under the Convention on International Civil Aviation (Chicago Convention), ICAO’s technical and diplomatic leadership has supported the development of a safe, secure, efficient and dependable global air transport network that connects people, cultures and businesses, advances socio-economic prosperity, and contributes to the achievement of the United Nations Sustainable Development Goals. As aviation undergoes rapid digital and technological transformation, ICAO continues to adapt its work and expand partnerships across the United Nations system and with technical stakeholders to advance a strategic global vision and effective, sustainable solutions for the future of international civil aviation.

The Office of Internal Oversight (OIO) is responsible for the systematic evaluation and internal audit of ICAO's programmes, projects, and activities, with a view to assessing whether the Organization's stated objectives and impacts have been achieved, and resources have been expended in accordance with applicable rules and regulations and with due regard to economy, efficiency and effectiveness. The Office provides an annual report of its evaluations and internal audits to the ICAO Council.

Evaluation is mainly focused on assessing the relevance, effectiveness, efficiency, impact and sustainability of programmes and activities against established plans and objectives. Internal audit ensures that a system of cost-effective internal controls has been established to mitigate potential risks to the Organization, to ensure compliance with policies, regulations, rules and administrative directives, and to ensure accountability for the proper, efficient and economical use of resources, consistent with organizational objectives and applicable best practices.

Major duties and Responsibilities

Under the general guidance of the Chief, Office of Internal Oversight (OIO), and reporting to the Evaluation Specialist, the consultant will contribute to the independent mid‑term evaluation of ICAO’s 2026–2028 Business Plan. The evaluation will assess the Plan’s relevance, coherence, effectiveness, efficiency, emerging results and prospects for sustainability; examine the adequacy of its design, alignment with ICAO’s Strategic Plan, governance, resourcing, monitoring, risk‑management and reporting arrangements; and generate timely evidence, lessons and actionable recommendations to strengthen implementation during the remainder of the planning cycle and inform future corporate planning.

Function 1 (incl. Expected results)

Prepares the inception report, evaluability assessment and evaluation design

Following An Initial Document Review And Consultations With Intended Users, The Consultant, With The Support Of The Evaluation Specialist Will Prepare An Inception Report That Operationalizes The TOR. The Work Will Apply a Theory‑based, Utilization‑focused And Mixed‑methods Design And Will:

• Reconstruct and validate the Business Plan theory of change, results chain, assumptions, dependencies and external factors;
• Assess evaluability, including the completeness and quality of baselines, targets, indicators, data sources, financial linkages, reporting frequency and disaggregation;
• Map Strategic Goals, High‑Priority and Supporting Enablers, outcomes, outputs, programmes, projects, activities, resources, accountable entities and corporate risks;
• Finalize the evaluation matrix, judgment criteria, sampling strategy, stakeholder map, methods, data‑quality plan, ethics protocol, limitations, work plan and report outline; and
• Design and pilot data collection instruments and interview or focus‑group protocols.

Function 2 (incl. Expected results)

Collects, validates and analyses quantitative and qualitative evidence

The Consultant With The Support Of The Evaluation Specialist, Will Review The Business Plan, Strategic Plan, Operating Plans, Results And Resource Frameworks, Governance And Risk Documentation, Performance And Financial Data, Prior Evaluations And Audits, Assembly And Council Documentation, And Other Relevant Sources. The Team Will Combine Portfolio‑wide Analysis With a Purposive, Risk‑based Sample Covering The Six Strategic Goals, Enablers, Regions, Funding And Implementation Status, Delivery Modalities, Data Quality And Strategic Significance. The Consultant Will:

• Compile and validate a source and data inventory linking Business Plan elements, indicators, resources, accountable entities and risks;
• Collect evidence through document review, analysis of ICAO systems and datasets, interviews, focus groups, surveys, process walkthroughs and selected comparative case studies;
• Prepare accurate summary interview notes and case‑study or portfolio analytical files that provide an auditable evidence trail;
• Triangulate evidence, assess its strength, analyse progress against baselines, milestones and expected trajectories, and distinguish verified findings, stakeholder perceptions and evidence gaps;
• Assess contribution rather than overstate attribution, considering implementation pathways, assumptions, dependencies, alternative explanations and contextual factors; and
• Prepare and present emerging findings for validation and OIO quality assurance.

Function 3 (incl. Expected results)

Prepares the draft and final evaluation products and formulates prioritized, actionable recommendations

The Consultant, With The Support Of The Evaluation Specialist, Will Synthesize The Evidence Into a Concise, Balanced And Decision‑oriented Evaluation Report That Responds To The Evaluation Questions And Links Findings, Conclusions And Recommendations. The Consultant Will:

• Prepare a draft report organized by evaluation criterion, with evidence‑based findings, conclusions, lessons, good practices and prioritized recommendations;
• Develop recommendations specifying rationale, priority, responsible entity, indicative timeframe, resource implications and proposed closing criteria;
• Present the draft findings and facilitate validation with relevant stakeholders while safeguarding the evaluation’s independence;
• Maintain a comment matrix showing how factual corrections and substantive comments were considered and addressed;
• Finalize the evaluation report, executive summary, recommendation tracking table and presentation materials for senior management, EAAC and/or the Council, as required.

Function 4 (incl. Expected results)

Performs Other Duties, As Assigned

Result of Service

• Inception report, including reconstructed theory of change, evaluability assessment, evaluation matrix, sampling strategy, stakeholder map, methods, data collection instruments, data-quality and ethics protocols, limitations, work plan and report outline – draft and final;
• PowerPoint presentation on emerging findings or validation workshop materials;
• Mid‑term evaluation report following OIO outline and template, normally not exceeding 50 pages excluding annexes – draft and final;
• Comment matrix recording the disposition of stakeholder comments;

Qualifications And Experience

Educational

An advanced level university degree (Masters’ degree or equivalent) in management, evaluation, economics, social science, an aviation‑related field, or any other related field is required. A first‑level university degree in combination with additional years of qualifying experience may be accepted in lieu of the advanced university degree.

Professional experience and knowledge

Essential:

• A minimum of ten years of progressively responsible professional experience in evaluation, including experience leading complex corporate, strategic, policy or organizational evaluations;
• Demonstrated expertise in theory‑based, utilization‑focused and mixed‑methods evaluation, including evaluability assessment, theories of change, contribution analysis, portfolio analysis and purposive sampling;
• Strong quantitative and qualitative analytical skills, including assessment of results and resources frameworks, indicators, baselines, targets, financial and performance data, governance, risk and reporting systems;
• Experience conducting evaluations in the United Nations system or another multilateral or international organization, with sound knowledge of UNEG Norms and Standards;
• Proven ability to engage senior management, governing bodies, technical specialists, regional offices and diverse external stakeholders while maintaining independence and confidentiality;
• Excellent facilitation, presentation and report‑writing skills in English; and
• Demonstrated ability to deliver high‑quality, evidence‑based products within strict deadlines.

Desirable:

• Knowledge of strategic planning, results‑based management, budgeting, enterprise risk management and organizational performance frameworks;
• Experience evaluating Strategic Plans, Business Plans or Corporate Strategic Objectives or goals;
• Knowledge of ICAO, international civil aviation, the Chicago Convention, ICAO’s Strategic Goals, and the roles of Member States, Regional Offices and governing bodies.

Language Skills

Essential:

• Fluent reading, writing and speaking abilities in English

Desirable:

• A working knowledge of a second language of the Organization (Arabic, Chinese, French, Russian, or Spanish)

Conditions of Employment

The consultant will be held to the highest ethical standards and will be required to sign a Code of Conduct and declarations covering independence, conflicts of interest and confidentiality upon acceptance of the assignment. The evaluation will be conducted in accordance with UNEG Norms and Standards and applicable ICAO evaluation requirements. The consultant must safeguard sensitive information, support informed participation, maintain evidence traceability, mitigate bias and preserve the independence of evaluative judgments.

T he selected consultant is expected to be employed within the period of 2 December 2026 – 30 June 2027 for 55 working days.