Senior AI Foundations Engineer – Scalable LLM & AI Systems

Capital One National Association; Cambridge (MA); On-site; USD 197,000 - 225,000; Full time

Job summary

Capital One is seeking an AI Engineer 4 to join the IFX team in Cambridge, MA, building responsible AI systems and production-ready pipelines. You will design and deploy AI components, collaborate across disciplines, and advance state-of-the-art techniques on a scalable platform.

The role emphasizes performance, governance, and observability as you drive end-to-end AI capabilities for real-world banking applications.

Qualifications

• Bachelor's degree in Computer Science or related field plus at least 4 years of experience developing AI/ML algorithms or technologies, or a Master's degree plus at least 2 years of experience.
• At least 4 years of experience programming with Python, Go, Scala, CUDA, or Java.
• Experience leading development AI systems with tradeoffs around cost, latency, throughput and accuracy.
• 6 years of experience deploying scalable and responsible AI solutions on cloud platforms (e.g. AWS, Google Cloud, Azure, or equivalent private cloud)
• Experience developing AI and ML algorithms or technologies (e.g. LLM Inference, Similarity Search and VectorDBs, Guardrails, Memory) using Python, C++, C#, Java, CUDA, or Golang

Responsibilities

• Partner with a cross-functional team of engineers, research scientists, technical program managers, and product managers to deliver AI-powered products.
• Design, develop, test, deploy, and support AI software components including foundation model training, large language model inference, agents and multi-agent workflows, similarity search, guardrails, model evaluation, experimentation, governance, and observability.
• Leverage a broad stack of Open Source and SaaS AI technologies such as AWS Ultraclusters, Huggingface, VectorDBs, PyTorch, and more.
• Invent and introduce state-of-the-art foundation model optimization techniques to improve performance, scalability, cost, latency, throughput.
• Contribute to the technical vision and the long term roadmap of foundational AI systems.
• Own the end-to-end architecture for complex AI systems - ensuring maintainability, observability, and ethical alignment
• Define and maintain service-level objectives (SLOs) for AI reliability, including latency, uptime, and model performance drift
• Collaborate with infrastructure engineering to optimize GPU/TPU utilization and accelerate model inference pipelines
• Lead cross-functional technical reviews for new AI system deployments, ensuring security, data governance, and compliance standards are met
• Mentor Principal and Senior Associates on scalable design, performance tuning and research-to-production translation

Skills

Python

Go

Scala

CUDA

Java

Education

Bachelor's degree in Computer Science or related fields

Master's degree in Computer Science or related fields

Tools

AWS

Google Cloud

Azure

HuggingFace

VectorDBs

PyTorch

Job description

Capital One is seeking an AI Engineer 4 to join the IFX team in Cambridge, MA, building responsible AI systems and production-ready pipelines. You will design and deploy AI components, collaborate across disciplines, and advance state-of-the-art techniques on a scalable platform.

The role emphasizes performance, governance, and observability as you drive end-to-end AI capabilities for real-world banking applications.