Senior AI Engineer: LLM Inference & FM Hosting

Capital One; McLean (VA); On-site; USD 197,000 - 225,000; Full time

Job summary

Capital One is seeking a Lead AI Engineer (FM Hosting, LLM Inference) to build responsible, scalable AI systems. You will partner with cross-functional teams to deliver AI-powered products and supervise foundation model training, LLM inference, and guardrails within a high-performance AI infrastructure.

The role requires strong software engineering and ML/AI expertise, with hands-on experience in Python, Java, and cloud platforms.

Qualifications

• Bachelor’s degree in Computer Science, AI, Electrical Engineering, Computer Engineering, or related fields plus at least 4 years of experience developing AI and ML algorithms or technologies, or a Master’s degree in Computer Science, AI, Electrical Engineering, Computer Engineering, or related fields plus at least 2 years of experience developing AI and ML algorithms or technologies
• At least 4 years of experience programming with Python, Go, Scala, or Java

Responsibilities

• Partner with a cross-functional team of engineers, research scientists, technical program managers, and product managers to deliver AI-powered products that change how our associates work and how our customers interact with Capital One.
• Design, develop, test, deploy, and support AI software components including foundation model training, large language model inference, similarity search, guardrails, model evaluation, experimentation, governance, and observability, etc.
• Leverage a broad stack of Open Source and SaaS AI technologies such as AWS Ultraclusters, Huggingface, VectorDBs, Nemo Guardrails, PyTorch, and more.
• Invent and introduce state-of-the-art LLM optimization techniques to improve the performance scalability, cost, latency, throughput of large scale production AI systems.
• Contribute to the technical vision and the long term roadmap of foundational AI systems at Capital One.

Skills

Python development

ML engineering

Cloud platforms

Education

Bachelor’s degree in CS/AI/EE/CE or related fields

Master’s degree in CS/AI/EE/CE or related fields

Tools

Python

Go

Scala

Java

C

C#

Job description

Capital One is seeking a Lead AI Engineer (FM Hosting, LLM Inference) to build responsible, scalable AI systems. You will partner with cross-functional teams to deliver AI-powered products and supervise foundation model training, LLM inference, and guardrails within a high-performance AI infrastructure.

The role requires strong software engineering and ML/AI expertise, with hands-on experience in Python, Java, and cloud platforms.