Hybrid AI Evaluation Engineer - CI/CD & Tests

Citigroup Inc.; EUA; Teletrabalho; USD 140 000 - 190 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Hybrid work

Health insurance

Pension plan

Parental leave

Resumo da oferta

Citigroup is seeking a Forward Deployed Engineer to build automated evaluation harnesses and testbeds for production AI agents. You will optimize prompts, JSON schemas, and regression datasets, ensuring reliable, safe agent behavior across CI/CD pipelines. Strong Python, API, and Docker skills are essential.

You will collaborate with LOB domain experts and senior engineers, curating datasets and automating tests, while operating with autonomy and focusing on compliance and data integrity.

Qualificações

• Bachelor's degree in Computer Science, Data Science, or related quantitative discipline.
• Proven experience in Python development and AI tooling.
• Experience with REST/gRPC APIs and containerized deployments.

Responsabilidades

• Construct automated evaluation harnesses and regression testbeds.
• Optimize prompts, output schemas, and JSON parsers for deterministic agent behavior.
• Build and maintain automated test pipelines in CI/CD environments.

Conhecimentos

Python

REST APIs

Docker

Git workflows

AI eval frameworks

CI CD pipelines

Formação académica

Bachelor's degree in CS/Math/Data Science

Ferramentas

Agentflow Studio

Lightspeed

Tekton

Jenkins

Descrição da oferta de emprego

Citigroup is seeking a Forward Deployed Engineer to build automated evaluation harnesses and testbeds for production AI agents. You will optimize prompts, JSON schemas, and regression datasets, ensuring reliable, safe agent behavior across CI/CD pipelines. Strong Python, API, and Docker skills are essential.

You will collaborate with LOB domain experts and senior engineers, curating datasets and automating tests, while operating with autonomy and focusing on compliance and data integrity.