AI Data Engineer: Real-Time Pipelines for LLMs

Capital One; New York (NY); On-site; USD 251,000 - 286,000; Full time

Job summary

Capital One is seeking a Data Engineer to shape the AI-enabled data ecosystem, bridging enterprise data architecture with AI workloads. You will design scalable pipelines, real-time architectures, and feature stores, collaborating with data scientists and executives to produce production-ready data solutions.

The role emphasizes governance, security, and cost efficiency across multi-cloud environments, with a focus on LLM workflows and RAG pipelines.

Qualifications

• Bachelor's degree in CS or related quantitative field.
• 6+ years of application development experience.
• 4+ years distributed data experience.
• 4+ years SQL experience.
• 4+ years programming in Python, Java, or Scala.
• 4+ years designing and developing data pipelines.
• 2+ years data modeling with relational and non-relational databases.

Responsibilities

• Architect and scale batch and real-time data pipelines for AI workloads.
• Lead architectural blueprint and mentoring for engineers.
• Implement data governance, privacy safeguards, and lineage tracking.
• Build scalable feature stores for real-time inference and offline training.
• Optimize cross-cloud pipelines for performance and cost.
• Collaborate with PMs, data scientists, and executives to deliver production-ready solutions.

Skills

Distributed data

SQL

Python

Java

Scala

Data pipelines

Education

Bachelor's Degree in Computer Science or related field

Master's Degree (preferred)

Tools

Snowflake

AWS

Azure

GCP

Job description

Capital One is seeking a Data Engineer to shape the AI-enabled data ecosystem, bridging enterprise data architecture with AI workloads. You will design scalable pipelines, real-time architectures, and feature stores, collaborating with data scientists and executives to produce production-ready data solutions.

The role emphasizes governance, security, and cost efficiency across multi-cloud environments, with a focus on LLM workflows and RAG pipelines.