Senior AI Engineer: Foundation Models & LLM Systems

Capital One; New York (NY); On-site; USD 140,000 - 200,000; Full time

Benefits offered by this job

Health, financial, and other benefits

Performance-based incentivecomp

Job summary

Capital One seeks an AI Engineer to design and deliver AI systems focused on benchmarking, evaluation, explainability, and scalable foundations for production LLM applications. You will own end-to-end architecture and partner across teams to ensure reliability, observability, and ethical alignment.

The role emphasizes collaboration with engineers, researchers, PMs, and product teams, with exposure to hardware accelerators (GPU/TPU) and cloud services.

Qualifications

• Bachelor's or Master's degree in CS/AI required.
• 4+ years of AI/ML development experience.
• Strong programming skills in Python, Go, Scala, CUDA or Java.

Responsibilities

• Collaborate with engineers, research scientists, TPMs, and PMs to deliver AI-powered products for associates and customers.
• Design, develop, test, deploy, and support AI components including foundation model training, LLM inference, agents and multi-agent workflows, similarity search, guardrails, model evaluation, experimentation, governance, and observability.
• Use a broad stack of open source and SaaS AI technologies such as AWS Ultraclusters, Huggingface, VectorDBs, and PyTorch, along with cloud services including AWS, Google Cloud, and Azure.
• Introduce state-of-the-art optimization techniques to improve performance and production efficiency, focusing on scalability, cost, latency, and throughput.
• Own end-to-end architecture for complex AI systems, ensuring maintainability, observability, and ethical alignment.
• Define and maintain service-level objectives (SLOs) for AI reliability, including latency, uptime, and model performance drift.
• Partner with infrastructure engineering to optimize GPU/TPU utilization and accelerate inference pipelines.
• Lead cross-functional technical reviews for new AI system deployments, supporting security, data governance, and compliance standards.
• Mentor Principal and Senior Associates on scalable design, performance tuning, and research-to-production translation.

Education

Bachelor's Degree in Computer Science / AI or related field

Master's Degree in Computer Science / AI or related field

Tools

Python

Go

Scala

CUDA

Java

Job description

Capital One seeks an AI Engineer to design and deliver AI systems focused on benchmarking, evaluation, explainability, and scalable foundations for production LLM applications. You will own end-to-end architecture and partner across teams to ensure reliability, observability, and ethical alignment.

The role emphasizes collaboration with engineers, researchers, PMs, and product teams, with exposure to hardware accelerators (GPU/TPU) and cloud services.