Senior AI Production Engineer: Forward-Deploy Lead

Citi; New York (NY); On-site; USD 121,000 - 182,000; Full time

Job summary

The Applied AI and Agents Factory at Citi is seeking engineers to turn advanced AI capabilities into production-ready solutions. You’ll work with business and technology teams to tackle complex challenges, build intelligent agents, and take ideas from concept through live deployment.

This role pushes beyond conversational tools, focusing on reasoning, planning, and multi-step workflows. You’ll own agent architecture, orchestration, enterprise integration, security, and observability across the

Qualifications

• Bachelor’s degree in Computer Science, Data Science, or related quantitative discipline.
• 5+ years of relevant software engineering or systems analysis experience (or 2+ years with specialized focus in Python and Generative AI application development).
• Solid Python programming proficiency, with practical experience integrating REST/gRPC APIs, Docker containers, and Git workflows.
• Strong working knowledge of AI evaluation frameworks, benchmarking methodologies (BLEU, ROUGE, semantic similarity, LLM-as-a-judge), and automated testing suites.
• Experience in managing and implementing automated test pipelines in CI/CD environments (Lightspeed, Tekton, Jenkins).
• Ability to work effectively under pressure, managing tight sprint deadlines and unexpected changes in model outputs or LOB requirements.
• Good written and verbal communication skills with a methodical, detail-oriented approach to software quality and data integrity.

Responsibilities

• Constructs automated evaluation test suites using Agentflow Studio and CI/CD pipelines.
• Curates golden datasets, synthetic benchmarks, and regression testbeds with domain experts and senior engineers.
• Improves prompts, system instructions, few-shot examples, and JSON parsers for deterministic agent behavior.
• Maintains automated test execution pipelines and gates prior to staging and production rollout.
• Analyzes complex agent errors and performance to suggest advanced coding and prompt solutions.
• Implements security measures and Lakera Guardrail policies in post-implementation analysis.
• Monitors evaluation cycles and provides reports to stakeholders and squad leads.
• Operates with independence and acts as an evaluation SME within the pod team.
• Assesses risk and ensures compliance with laws, regulations, and Citi policies.

Skills

Python proficiency

AI evaluation frameworks

CI/CD experience

Communication skills

Work under pressure

Software engineering experience

Education

Bachelor’s in Computer Science or related

Tools

REST APIs

Docker

Git workflows

Lightspeed

Tekton

Jenkins

Job description

The Applied AI and Agents Factory at Citi is seeking engineers to turn advanced AI capabilities into production-ready solutions. You’ll work with business and technology teams to tackle complex challenges, build intelligent agents, and take ideas from concept through live deployment.

This role pushes beyond conversational tools, focusing on reasoning, planning, and multi-step workflows. You’ll own agent architecture, orchestration, enterprise integration, security, and observability across the