AI Foundations Engineer — Scalable LLMs & Guardrails

Capital One; Cambridge (MA); On-site; USD 197,000 - 225,000; Full time

Job summary

Capital One is seeking an AI Engineer 4 to lead the design and deployment of AI foundations across production systems in Cambridge, MA. You will collaborate with engineers, scientists, PMs, and product leads to ship AI-powered products that transform how customers and associates interact with Capital One.

You will own end-to-end architecture for complex AI systems, optimize performance and cost, set SLOs, and mentor senior engineers while advancing responsible AI practices and governance.

Qualifications

• Bachelor's degree in CS/AI/EE/CE with at least 4 years of AI/ML experience.
• Master's degree in CS/AI/EE/CE with at least 2 years of AI/ML experience.
• At least 4 years of experience programming with Python, Go, Scala, CUDA, or Java.

Responsibilities

• Partner with cross-functional team to deliver AI-powered products that change how associates work and how customers interact with Capital One.
• Design, develop, test, deploy, and support AI software components including foundation model training, LLM inference, agents and multi-agent workflows, similarity search, guardrails, model evaluation, experimentation, governance, and observability.
• Leverage a broad stack of Open Source and SaaS AI technologies such as AWS Ultraclusters, Huggingface, VectorDBs, PyTorch, and more.
• Invent and introduce state-of-the-art foundation model optimization techniques to improve performance — reliability, latency, throughput, and cost.
• Contribute to the technical vision and long-term roadmap of foundational AI systems at Capital One.
• Own the end-to-end architecture for complex AI systems, ensuring maintainability, observability, and ethical alignment.
• Define and maintain service-level objectives (SLOs) for AI reliability, including latency and uptime.
• Collaborate with infrastructure to optimize GPU/TPU utilization and accelerate model inference pipelines.
• Lead cross-functional technical reviews for new AI deployments, ensuring security, data governance, and compliance.
• Mentor Principal and Senior Associates on scalable design, performance tuning and research-to-production translation.

Skills

Foundation model training

LLM inference

Agents workflows

Similarity search

Guardrails

Model evaluation

Experimentation

Governance

Observability

Education

Bachelor's degree in CS/AI/EE/CE

Master's degree in CS/AI/EE/CE

Tools

AWS

GCP

Azure

Job description

Capital One is seeking an AI Engineer 4 to lead the design and deployment of AI foundations across production systems in Cambridge, MA. You will collaborate with engineers, scientists, PMs, and product leads to ship AI-powered products that transform how customers and associates interact with Capital One.

You will own end-to-end architecture for complex AI systems, optimize performance and cost, set SLOs, and mentor senior engineers while advancing responsible AI practices and governance.