Senior AI Data Engineer — Real-Time & LLM Pipelines

DataJobs; Richmond (VA); On-site; USD 209,000 - 239,000; Full time

Job summary

Capital One’s Capital One Enterprise Platforms Technology team is building an AI-enabling data ecosystem that can support modern machine learning and Large Language Model workloads. As a Data Engineer 5, you will help design and deliver resilient, scalable batch and real-time pipelines and streaming architectures that turn executive data needs into production-ready systems.

This role is based in Richmond, VA (onsite).

Qualifications

• Bachelor’s Degree or higher in Computer Science or related quantitative field.
• 6+ years of experience in application development.
• 4+ years of experience in distributed data.
• 4+ years of experience with SQL.
• 4+ years programming with Python, Java, or Scala.
• 4+ years designing and developing data pipelines.
• 2+ years in data modeling and end-to-end data solutions using relational and non-relational databases.

Responsibilities

• Architect, build, and scale latency-optimized data pipelines for batch and real-time use cases.
• Design pipelines for structured, semi-structured, and unstructured data for AI/ML applications and LLMs.
• Help define the architectural blueprint for the Top of House AI data layer.
• Act as hands-on tech lead mentoring junior and mid-level engineers on coding standards.
• Collaborate with Enterprise Security and Data Governance on data classification, privacy safeguards, and automated lineage tracking.
• Build and maintain scalable feature stores for real-time inferencing and offline training.
• Audit and optimize cross-cloud data pipelines, query performance, and storage infrastructure for efficiency and reliability.
• Work with TPMs, DSs, Top of House leads, and C-suite stakeholders to translate executive data requirements into production-ready solutions.

Skills

Python

SQL

Databricks

Snowflake

Java

Scala

AWS

Microsoft Azure

Google Cloud

Education

Bachelor’s Degree or higher

Master’s Degree

Tools

Python

SQL

Databricks

Snowflake

Java

Scala

AWS

Microsoft Azure

Google Cloud

Job description

Capital One’s Capital One Enterprise Platforms Technology team is building an AI-enabling data ecosystem that can support modern machine learning and Large Language Model workloads. As a Data Engineer 5, you will help design and deliver resilient, scalable batch and real-time pipelines and streaming architectures that turn executive data needs into production-ready systems.

This role is based in Richmond, VA (onsite).