Senior AI Quality Engineer - Safety & RAG (Hybrid, AVP)

Citi; New York (NY); Hybrid; USD 140,000 - 210,000; Full time

Job summary

Citi is seeking an AI Quality Engineer to lead testing across Agentic AI, RAG pipelines, and safety modules. The role focuses on ensuring the AI systems are safe, accurate, explainable, and production-ready at scale. You will work on testing strategies, automation, and reliability for complex autonomous AI solutions.

You will collaborate with risk, engineering, and MLOps teams to implement robust validation, monitoring, and governance practices for enterprise AI deployments.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or related field.
• 8+ years of software or AI/ML quality engineering experience.
• Hands-on experience with RAG systems or agentic AI is required.
• Strong background in performance testing and AI safety/security assessments.

Responsibilities

• Design and execute end-to-end test strategies for Agentic AI pipelines.
• Validate agent reasoning, planning, and decision-making chains.
• Test tool-use correctness and agent memory systems across sessions.
• Develop automated evaluation pipelines integrated into CI/CD workflows.
• Create data quality and validation frameworks for training, retrieval, and inference data.
• Establish observability dashboards and SLAs for AI systems.

Skills

Agentic AI testing

RAG testing

Test automation

AI safety testing

Performance testing

Education

Bachelor's degree in CS/Engineering

Master's degree a plus

Tools

Great Expectations

Deequ

Job description

Citi is seeking an AI Quality Engineer to lead testing across Agentic AI, RAG pipelines, and safety modules. The role focuses on ensuring the AI systems are safe, accurate, explainable, and production-ready at scale. You will work on testing strategies, automation, and reliability for complex autonomous AI solutions.

You will collaborate with risk, engineering, and MLOps teams to implement robust validation, monitoring, and governance practices for enterprise AI deployments.