Senior AI Systems Engineer: LLM Inference & Production AI

Capital One; New York (NY); On-site; USD 170,000 - 230,000; Full time

Job summary

Capital One is seeking a Sr. Lead AI Engineer (FM Hosting, LLM Inference) to advance responsible AI across banking.

You will partner with engineers, researchers, and product teams to design and scale AI-powered experiences and infrastructure. The role focuses on foundation models, inference, guardrails, and observability, using PyTorch, Hugging Face, and vector databases to deliver production-ready AI at scale.

Qualifications

• Bachelor's degree and 6+ years of AI/ML experience.
• Master's degree and 4+ years of AI/ML experience.
• Strong Python/Scala/Java programming background.
• Experience with cloud AI deployments and model inference.

Responsibilities

• Partner with cross-functional teams to deliver AI-powered products.
• Design, develop, test, deploy, and support AI software components including LLM inference and guardrails.
• Lead research and productionization of AI systems, optimizing performance and cost.
• Contribute to the long-term roadmap of foundational AI systems at Capital One.
• Mentor engineers and influence stakeholders.

Skills

Python

Go

Scala

Java

Education

Bachelor's degree in Computer Science, AI, Electrical Engineering, Computer Engineering, or related fields

Master's degree in the same fields

Tools

AWS Ultraclusters

Hugging Face

VectorDBs

Nemo Guardrails

PyTorch

Job description

Capital One is seeking a Sr. Lead AI Engineer (FM Hosting, LLM Inference) to advance responsible AI across banking.

You will partner with engineers, researchers, and product teams to design and scale AI-powered experiences and infrastructure. The role focuses on foundation models, inference, guardrails, and observability, using PyTorch, Hugging Face, and vector databases to deliver production-ready AI at scale.