AI Systems Engineer — LLM Inference & Orchestration

Capital One; New York (NY); On-site; USD 251,000 - 286,000; Full time

Job summary

Capital One in New York, NY seeks an AI Engineer 5 to join the Intelligent Foundations and Experiences team. You will design, build, and scale AI systems across LLM inference, foundation models, and multi-model orchestration, partnering with engineers, researchers, and product managers.

You’ll optimize performance and cost, leverage PyTorch, HuggingFace, VectorDBs, and cloud services, and help shape the technical roadmap for responsible AI at Capital One.

Qualifications

• Bachelor's or Master's in a relevant field with required years of experience.
• 6+ years of experience developing AI/ML algorithms or technologies.
• 6+ years of programming with Python, Go, Scala, CUDA, or Java.
• Experience leading development of AI systems and cloud deployment.

Responsibilities

• Deliver AI-powered products with cross-functional teams.
• Design, develop, test, deploy AI software components including foundation model training and LLM inference.
• Leverage Open Source and SaaS AI tech such as AWS Ultraclusters, HuggingFace, VectorDBs, PyTorch.
• Optimize techniques to improve performance, scalability, latency, and cost of large-scale AI systems.
• Contribute to the technical vision and long-term roadmap for foundational AI systems.
• Design and optimize multi-model orchestration pipelines and governance reviews.

Skills

Python

Go

Scala

CUDA

Java

ML algorithms

AI systems

Model inference

Education

Bachelor’s degree in Computer Science, AI, Electrical Engineering, Computer Engineering, or related fields + 6+ years experience

Master’s degree in Computer Science, AI, Electrical Engineering, Computer Engineering, or related fields + 4+ years experience

Tools

AWS

Google Cloud

Azure

HuggingFace

PyTorch

VectorDBs

Job description

Capital One in New York, NY seeks an AI Engineer 5 to join the Intelligent Foundations and Experiences team. You will design, build, and scale AI systems across LLM inference, foundation models, and multi-model orchestration, partnering with engineers, researchers, and product managers.

You’ll optimize performance and cost, leverage PyTorch, HuggingFace, VectorDBs, and cloud services, and help shape the technical roadmap for responsible AI at Capital One.