Senior Data Engineer for AI Data Platform (Python, SQL)

Capital One Group; New York, Northern (NY, KY); Hybrid; USD 251,000 - 286,000; Full time

Job summary

Capital One is seeking a Data Engineer to design and implement an AI-enabled data ecosystem, delivering scalable batch and real-time pipelines for LLM workflows and RAG pipelines. You will bridge traditional data architecture with modern AI capabilities, guiding best practices and collaborating with scientists, analysts, and other engineers.

You will own architectural blueprints for the Top of House AI data layer, ensure data governance and security, build feature stores, optimize performance

Qualifications

• Bachelor's degree or higher in Computer Science or related quantitative field.
• 6+ years of experience in application development.
• 4+ years of experience in distributed data and SQL.
• 4+ years of experience programming in Python/Java/Scala.
• 2+ years of data modeling and end-to-end data solutions.

Responsibilities

• Architect, build, and scale batch and real-time data pipelines for AI/ML applications.
• Lead and mentor junior engineers, enforcing coding standards and design patterns.
• Collaborate with Security and Governance teams for data classification and privacy safeguards.
• Build and maintain scalable feature stores for real-time and offline model training.
• Optimize cross-cloud pipelines, queries, and storage for cost and reliability.

Skills

Python

SQL

Distributed data

Real-time streaming

Education

Bachelor's degree in CS or related

Tools

Databricks

Snowflake

LLMs

Job description

Capital One is seeking a Data Engineer to design and implement an AI-enabled data ecosystem, delivering scalable batch and real-time pipelines for LLM workflows and RAG pipelines. You will bridge traditional data architecture with modern AI capabilities, guiding best practices and collaborating with scientists, analysts, and other engineers.

You will own architectural blueprints for the Top of House AI data layer, ensure data governance and security, build feature stores, optimize performance