Lead Data Engineer - Python, AWS, Spark & GenAI

Capital One National Association; Virginia (MN); On-site; USD 179,400 - 245,600; Full time

Job summary

Capital One is seeking a Lead Data Engineer to architect and deliver data pipelines using Python, SQL, Spark, Kafka, Snowflake and Databricks. You will collaborate with Agile teams, mentor peers, and drive performance-optimized cloud solutions that empower financial services customers.

The role emphasizes data modeling, governance, and real-time streaming experience within a large data platform, with multiple US locations and strong growth opportunities.

Qualifications

• Bachelor’s Degree required with 4+ years in application development.
• Minimum 2 years in big data technologies.
• At least 1 year of cloud computing experience (AWS/Azure/Google).
• Strong Python, SQL, Spark and data warehousing skills.

Responsibilities

• Design, develop, test, and support data solutions across Agile teams.
• Lead developers in data movement, distributed computing, and full-stack systems.
• Use Python, SQL, Snowflake and Databricks for cloud data warehousing.
• Optimize data designs for end users and downstream consumers.
• Mentor engineers, stay current with tech trends, and share learnings.
• Collaborate with product managers to deliver cloud-based experiences.
• Perform unit tests and code reviews for performance-tuned solutions.

Skills

Python

SQL

Spark

Kafka

Snowflake

Databricks

GenAI

AWS

Education

Bachelor's Degree

Master's Degree

Job description

Capital One is seeking a Lead Data Engineer to architect and deliver data pipelines using Python, SQL, Spark, Kafka, Snowflake and Databricks. You will collaborate with Agile teams, mentor peers, and drive performance-optimized cloud solutions that empower financial services customers.

The role emphasizes data modeling, governance, and real-time streaming experience within a large data platform, with multiple US locations and strong growth opportunities.