Senior Lead Data Engineer - Real-Time Streaming Platform

Capital One Group; Plano (TX); On-site; USD 209,000 - 239,000; Full time

Job summary

Capital One is seeking a Senior Lead Data Engineer to drive real-time data platforms and streaming pipelines in a cloud environment. You will lead a team of engineers building scalable data pipelines using Spark, Flink, Kafka, Snowflake and Redshift while ensuring best practices in data architecture and security.

The role requires deep expertise in Java/Python/Scala, plus hands-on experience with cloud services (AWS, Azure, Google Cloud).

Qualifications

• Bachelor's degree in a relevant field required.
• At least 6 years of experience in application development (internship not counted).
• At least 2 years of experience with big data technologies (Spark, Flink, Kafka, etc.).
• At least 1 year experience with cloud computing (AWS, Azure, or Google Cloud).

Responsibilities

• Lead design and build enterprise-level scalable streaming data platform.
• Build next-gen distributed streaming data pipelines and analytics stores.
• Lead engineers building data pipelines with Spark, Flink, Kafka, Snowflake, and Redshift.
• Influence best practices for data pipeline design and data architecture.
• Collaborate with Agile teams and ensure CI/CD and security alignment.
• Mentor team members and contribute to full-stack data solutions.

Skills

Java

Python

Scala

Spark

Flink

Kafka

Snowflake

Redshift

AWS

Education

Bachelor's degree

Master's degree

Tools

Spark

Flink

Kafka

Snowflake

Redshift

AWS

Job description

Capital One is seeking a Senior Lead Data Engineer to drive real-time data platforms and streaming pipelines in a cloud environment. You will lead a team of engineers building scalable data pipelines using Spark, Flink, Kafka, Snowflake and Redshift while ensuring best practices in data architecture and security.

The role requires deep expertise in Java/Python/Scala, plus hands-on experience with cloud services (AWS, Azure, Google Cloud).