Lead Data Engineer (Navigator platform Revenue operations)

Capital One; Plano (TX); On-site; USD 120,000 - 150,000; Full time

Job summary

A leading company is seeking a Lead Data Engineer for their Navigator platform. The role involves building scalable data pipelines, optimizing product pricing, and collaborating with teams to deliver cloud-based solutions. Ideal candidates will have extensive experience in big data technologies and cloud computing.

Qualifications

• 4+ years in application development excluding internships.
• 2+ years in big data technologies.
• 1+ year in cloud computing.

Responsibilities

• Lead design and build scalable, low-latency streaming data platforms.
• Build distributed streaming data pipelines using frameworks like Flink and Spark.
• Collaborate across teams to develop full-stack technical solutions.

Skills

Java

Python

SQL

Scala

Agile practices

Education

Bachelor’s Degree

Tools

AWS

Flink

Kafka

Spark

Snowflake

Redshift

DynamoDB

OpenSearch

Job description

Lead Data Engineer (Navigator platform Revenue operations)

Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, inclusive, and iterative delivery environment? At Capital One, you'll be part of a group of makers, breakers, doers, and disruptors, who solve real problems and meet real customer needs. We are seeking Data Engineers passionate about marrying data with emerging technologies. As a Capital One Lead Data Engineer, you’ll have the opportunity to drive a major transformation within Capital One.

The team works within the Capital One Navigator platform Revenue operations experience, developing tools, platforms, and products that facilitate seamless onboarding of dealers to the Navigator platform, generate revenue, and optimize product pricing within the platform.

They will join a world-class engineering team building SaaS products for car dealerships. The team is responsible for building highly available and resilient data pipelines that enable the platform/tools/products to operate efficiently, providing real-time data 100% of the time. They also manage data governance, monitor, and update data infrastructure.

What You’ll Do:

1. Lead design and build enterprise-level, scalable, low-latency, fault-tolerant streaming data platforms for meaningful insights.
2. Build next-generation distributed streaming data pipelines and analytics data stores using frameworks like Flink and Spark Streaming with languages such as Java, Scala, Python.
3. Lead engineers in building data pipelines with big data technologies (Spark, Flink, Kafka, Snowflake, AWS Big Data Services, Redshift) on medium to large datasets.
4. Influence best practices in data pipeline design, architecture, and processing of structured and unstructured data.
5. Work in a collaborative, agile environment focused on CI/CD, application resiliency, and security partnerships.
6. Share passion for technology trends, experimenting with new tech, participating in communities, and mentoring.
7. Collaborate across teams to design, develop, test, and support full-stack technical solutions.
8. Work with experienced developers in machine learning, microservices, and full-stack systems.
9. Partner with product managers to deliver cloud-based solutions that empower millions of Americans financially.
10. Conduct unit testing and reviews to ensure high-quality, performant code.

Basic Qualifications:

• Bachelor’s Degree
• At least 4 years in application development (internship experience excluded)
• At least 2 years in big data technologies
• At least 1 year in cloud computing (AWS, Azure, Google Cloud)

Preferred Qualifications:

• 7+ years in application development with Java, Python, SQL, or Scala
• 4+ years in a public cloud environment (AWS, Azure, Google Cloud)
• 4+ years with distributed data tools (Flink, Kafka, Spark)
• 4+ years working on real-time data and streaming applications
• 4+ years with NoSQL databases (DynamoDB, OpenSearch)
• 4+ years in data warehousing (Redshift, Snowflake)
• 4+ years UNIX/Linux experience including commands and scripting
• 2+ years in Agile practices

Note: Capital One will not sponsor employment authorization for this role at this time.