Forward Deployed Engineer - Avp

Citi Group

Greater London

Hybrid

GBP 110,000 - 150,000

Full time

3 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

27 days annual leave
Discretionary annual bonus
Private Medical Care & Life Insurance
Employee Assistance Program
Pension Plan
Paid Parental Leave
Special discounts for employees,Family

Job summary

Citi London is seeking an intermediate Forward Deployed Engineer to build automated evaluation harnesses and CI/CD tests for production AI agents. You will calibrate prompts, curate datasets, and optimize parsing schemas to ensure reliable, safe and efficient agent behavior.

Role focuses on Python application development, REST/gRPC API integration, Docker-based tooling, and automated testing within hybrid work arrangements.

Qualifications

  • Bachelor’s degree in Computer Science, Data Science, or related quantitative field.
  • Solid Python programming proficiency with REST/gRPC, Docker, and Git.
  • Experience with AI evaluation frameworks and automated testing.
  • Experience building automated test pipelines in CI/CD (Lightspeed, Tekton, Jenkins).
  • Ability to work under pressure and meet sprint deadlines.
  • Strong written and verbal communication for software quality and data integrity.

Responsibilities

  • Construct automated evaluation test suites with Agentflow Studio, measure metrics for agent tasks and latency.
  • Curate domain-specific datasets and regression testbeds in collaboration with senior engineers.
  • Optimize prompts, system messages, and JSON output parsers for deterministic behavior.
  • Maintain automated test pipelines and CI/CD gates before staging/production.
  • Analyze complex agent errors and recommend coding and prompting solutions.
  • Ensure compliance and safe usage with Lakera Guardrail policies.
  • Provide operational support and reporting to stakeholders.

Skills

Python programming
Python & APIs
AI evaluation frameworks
CI/CD pipelines
Time management
Communication skills

Education

Bachelor’s in CS

Tools

REST/gRPC APIs
Docker
Git workflows

Job description

About the Team: Build the next generation of agentic AI. The Applied AI and Agents Factory brings together ambitious engineers to turn advanced AI capabilities into production-ready solutions. Working directly with business and technology teams, you’ll tackle complex engineering challenges, build intelligent agents and take ideas from initial concept through to live deployment. This is an opportunity to work at the cutting edge of AI engineering, developing systems that go beyond conversational tools to reason, plan and execute multi-step workflows. You’ll build reliable agent workflows, integrate AI with enterprise systems and ensure solutions perform securely and consistently in production. With a strong focus on hands-on engineering, collaboration and rapid delivery, the Factory gives you the opportunity to solve challenging technical problems, develop your expertise and contribute reusable tools and engineering practices that support AI adoption across Citi. If you want to do more than experiment with AI - and build, deploy and scale systems that deliver real business value - this is where you can make your mark. About the Role: The Forward Deployed Engineer is an intermediate-level software engineering position responsible for constructing automated evaluation harnesses, curating regression testbeds, optimizing prompts and output schemas, and automating testing pipelines within embedded LOB delivery pods. The overall objective of this role is to ensure the reliability, accuracy, and safety of production AI agents by building rigorous evaluation suites in Agentflow Studio and driving automated CI/CD test execution.

Responsibilities
  • Conducts tasks related to the construction, calibration, and execution of automated evaluation test suites using Agentflow Studio, measuring agent task completion, tool call correctness, latency, and hallucination metrics.
  • Curates and maintains domain-specific golden datasets, synthetic benchmarks, and regression testbeds in collaboration with LOB domain experts and senior engineers.
  • Optimizes prompt templates, system instructions, few-shot examples, and structured JSON output parsers (Pydantic/JSON Schema) to maximize deterministic agent behavior.
  • Authors and maintains automated test execution and deployment pipelines within Lightspeed CI/CD, enforcing continuous evaluation gates prior to staging and production rollout.
  • Utilizes in-depth specialty knowledge of Python applications development to analyze complex agent errors, evaluate system performance, and recommend advanced coding and prompt solutions.
  • Recommends and implements security measures and Lakera Guardrail policies in post-implementation analysis of agent usage to ensure safe functionality.
  • Monitors and controls evaluation test cycles, providing operational support, performance metrics, and prompt diagnostic reporting to business stakeholders and squad leads.
  • Has the ability to operate with a limited level of direct supervision, exercising independence of judgement and autonomy while acting as an evaluation SME to pod team members.
  • Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm’s reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.
Qualifications
  • Relevant software engineering or systems analysis experience or a specialized focus in Python and Generative AI application development).
  • Solid Python programming proficiency, with practical experience integrating REST/gRPC APIs, Docker containers, and Git workflows.
  • Strong working knowledge of AI evaluation frameworks, benchmarking methodologies (BLEU, ROUGE, semantic similarity, LLM-as-a-judge), and automated testing suites.
  • Experience in managing and implementing automated test pipelines in CI/CD environments (Lightspeed, Tekton, Jenkins).
  • Ability to work effectively under pressure, managing tight sprint deadlines and unexpected changes in model outputs or LOB requirements.
  • Good written and verbal communication skills with a methodical, detail-oriented approach to software quality and data integrity.
Education
  • Bachelor’s/University degree or equivalent experience in Computer Science, Data Science, or related quantitative discipline.

What we’ll provide you This is a unique role that will put you in the position to be part of a new venture and actively drive change. Every day there will be new challenges that will help you develop new skills that can drive your career. By joining Citi London, you will not only be part of a business casual workplace with a hybrid working model (up to 2 days working at home per week), but also receive a competitive base salary (which is annually reviewed), and enjoy a whole host of additional benefits such as:

  • 27 days annual leave (plus bank holidays)
  • A discretional annual performance related bonus
  • Private Medical Care & Life Insurance
  • Employee Assistance Program
  • Pension Plan
  • Paid Parental Leave
  • Special discounts for employees, family, and friends

Visit our Global Benefits page to learn more. Alongside these benefits Citi is committed to ensuring our workplace is where everyone feels comfortable coming to work as their whole self, every day. We want the best talent around the world to be energized to join us, motivated to stay and empowered to thrive. #LI-AH4 ------------------------------------------------------

Job Family Group
Technology
Job Family
Applications Development
Time Type
Full time
Most Relevant Skills
Please see the requirements listed above.
Other Relevant Skills

For complementary skills, please see above and/or contact the recruiter.

Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.

If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi.

View Citi’s EEO Policy Statement and the Know Your Rights poster.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Forward Deployed Engineer - AVP
Forward Deployed Engineer - AVP

Citi Group • Greater London

Hybrid
GBP 90,000 - 120,000
27 days annual leave
Discretionary annual bonus
Private Medical Care & Life Insurance
+4
Forward Deployed Engineer - Avp
Forward Deployed Engineer - Avp

Citi • Greater London

Hybrid
GBP 80,000 - 110,000
Hybrid working model (up to 2 days at/
27 days annual leave
Discretionary annual bonus
+5
Senior Forward Deployed Ai Engineer - Vice President
Senior Forward Deployed Ai Engineer - Vice President

Citi Group • Greater London

Hybrid
GBP 120,000 - 160,000
27 days annual leave (plus bank-holidy
Discretional annual performance bonus
Private Medical Care & Life Insurance
+4
Forward Deployed Engineering - Director
Forward Deployed Engineering - Director

Citigroup Inc. • Greater London

Hybrid
GBP 180,000 - 240,000
27 days annual leave (plus bank hols)
Discretional annual bonus
Private Medical Care & Life Insurance
+4
Lead / Staff Forward Deployed Engineer (SVP)
Lead / Staff Forward Deployed Engineer (SVP)

Citi • Greater London

Hybrid
GBP 120,000 - 180,000
27 days annual leave
Discretional annual bonus
Private Medical Care & Life Insurance
+4
Lead / Staff Forward Deployed Engineer (Svp)
Lead / Staff Forward Deployed Engineer (Svp)

Citi • England

Hybrid
GBP 110,000 - 160,000
27 days annual leave (plus bank HOL)
Discretional annual bonus
Private Medical Care & Life Insurance
+4
Forward Deployed Engineering - Director
Forward Deployed Engineering - Director

Citi • England

Hybrid
GBP 170,000 - 230,000
Hybrid work model
Production AI Engineer - Vice President
Production AI Engineer - Vice President

Citi • Greater London

Hybrid
GBP 70,000 - 110,000
Hybrid work model
Private Medical Care & Life Insurance
Pension Plan
+4
Lead Ai Engineer - Enterprise Ai Products
Lead Ai Engineer - Enterprise Ai Products

Citi Group • Greater London

Hybrid
GBP 110,000 - 160,000
27 days annual leave
Discretionary annual bonus
Private Medical Care & Life Insurance
+4
Senior Forward Deployed Engineer - Vice President
Senior Forward Deployed Engineer - Vice President

Citi • Greater London

Hybrid
GBP 110,000 - 150,000
27 days annual leave
Discretional annual bonus
Private Medical Care & Life Insurance
+4