Senior Lead Data Engineer — Real-Time Streaming Leader

Socket.dev; Plano (TX); On-site; USD 209,000 - 239,000; Full time

Job summary

Capital One seeks a Senior Lead Data Engineer in Plano, TX to design and implement scalable streaming data platforms, drive real-time insights, and lead a team of engineers across big data technologies.

You will work with Java/Scala/Python, Flink, Spark, Kafka, Snowflake, Redshift, and AWS services, mentor engineers, and collaborate with product teams to deliver secure, resilient cloud-based data solutions.

Qualifications

• Bachelor's degree in a relevant field required.
• 6+ years of application development experience.
• 2+ years of big data technologies experience.
• 1+ year of cloud computing experience (AWS, Azure, Google Cloud).

Responsibilities

• Lead design and build enterprise level scalable, low-latency streaming data platform.
• Build the next generation distributed streaming data pipelines and analytics data stores using streaming frameworks (Flink, Spark Streaming) using Java, Scala, Python.
• Lead a group of engineers building data pipelines using big data technologies (Spark, Flink, Kafka, Snowflake, AWS Big Data Services, Snowflake, Redshift) on large datasets.
• Influence best practices for Data Pipeline design, Data architecture and processing of structured and unstructured data.
• Collaborate in an agile environment with CI/CD, resilience standards, and cyber/security partnerships.

Skills

Java

Scala

Python

Big data architectures

Cloud computing (AWS/Azure/GCP)

CI/CD practices

Leadership

Mentoring

Education

Bachelor’s degree in a related field

Master's degree

Tools

Flink

Spark

Kafka

Snowflake

Redshift

AWS

Job description

Capital One seeks a Senior Lead Data Engineer in Plano, TX to design and implement scalable streaming data platforms, drive real-time insights, and lead a team of engineers across big data technologies.

You will work with Java/Scala/Python, Flink, Spark, Kafka, Snowflake, Redshift, and AWS services, mentor engineers, and collaborate with product teams to deliver secure, resilient cloud-based data solutions.