Senior AI Engineer LLM & Foundation Models

Capital One Group; New York (NY); On-site; USD 215,000 - 246,000; Full time

Job summary

Capital One in New York, NY is seeking an AI Engineer 4 to help build responsible and scalable AI systems for real-time customer experiences. You will design and implement AI components, including foundation model training, inference, and governance, while collaborating across engineering, research, and product teams.

The role emphasizes optimization of performance, cost, and latency, with hands-on work on a broad stack (PyTorch, HuggingFace, VectorDBs) and strong governance and observability.

Qualifications

• Bachelor's or Master's degree in a relevant field with required experience.
• Proficiency in Python, Go, Scala, CUDA, or Java for AI/ML systems.
• Experience deploying scalable AI services on cloud platforms.

Responsibilities

• Partner with a cross-functional team to deliver AI-powered products.
• Design, develop, test, deploy and support AI components including training, inference, guardrails, evaluation and observability.
• Leverage Open Source and SaaS AI tech such as AWS Ultraclusters, HuggingFace, VectorDBs, and PyTorch.
• Invent state-of-the-art optimization techniques to improve performance, cost, latency, and throughput of large-scale AI systems.
• Contribute to the technical vision and long-term roadmap of foundational AI systems at Capital One.
• Own end-to-end architecture for complex AI systems ensuring maintainability, observability, and ethical alignment.
• Define and maintain service-level objectives (SLOs) for AI reliability, including latency and model performance drift.
• Collaborate with infrastructure to optimize GPU/TPU utilization and accelerate inference pipelines.
• Lead cross-functional technical reviews for new deployments with security and data governance in mind.
• Mentor Principal and Senior Associates on scalable design and production translation.

Skills

Python

Go

Scala

CUDA

Java

AI/ML algorithms

Distributed systems

Education

Bachelor's Degree in Computer Science, AI, Electrical Engineering, Computer Engineering, or related fields

Master's Degree in Computer Science, AI, Electrical Engineering, Computer Engineering, or related fields

Tools

AWS

Google Cloud

Azure

HuggingFace

PyTorch

VectorDBs

Guardrails

Job description

Capital One in New York, NY is seeking an AI Engineer 4 to help build responsible and scalable AI systems for real-time customer experiences. You will design and implement AI components, including foundation model training, inference, and governance, while collaborating across engineering, research, and product teams.

The role emphasizes optimization of performance, cost, and latency, with hands-on work on a broad stack (PyTorch, HuggingFace, VectorDBs) and strong governance and observability.