Senior Data Engineer Leader — Python, Spark & AWS

Capital One; McLean (VA); On-site; USD 197,000 - 225,000; Full time

Job summary

Capital One is seeking a Lead Data Engineer to drive data solutions across cloud and on‑prem environments. You will work with Python, Spark, and AWS to design and implement scalable data pipelines and ML workflows. You’ll mentor teammates, review code, and help shape architecture in a fast‑paced, collaborative setting.

The role requires strong hands-on experience in big data, distributed systems, and cloud platforms, with a proven track record delivering robust data products for customers.

Qualifications

• Bachelor's degree in a relevant field.
• 4+ years of application development (internship experience does not apply).
• 2+ years of experience with big data technologies.
• ≥1 year of experience with cloud computing (AWS, Azure, or Google Cloud).

Responsibilities

• Collaborate with Agile teams to design, develop, test, implement, and support technical solutions.
• Work with developers experienced in ML, distributed microservices, and full-stack systems.
• Use languages like Java, Scala, Python and RDBMS/NoSQL and cloud data warehousing services such as Redshift and Snowflake.
• Mentor other engineers and participate in technology communities.
• Collaborate with product managers to deliver cloud-based solutions for millions of users.
• Perform unit tests and code reviews to ensure performance and quality.

Skills

Python

SQL

Scala

Java

AI tooling

Big data concepts

Education

Bachelor's degree

Tools

Hadoop

Spark

Kafka

Redshift

Snowflake

EMR

MySQL

Mongo

Cassandra

Job description

Capital One is seeking a Lead Data Engineer to drive data solutions across cloud and on‑prem environments. You will work with Python, Spark, and AWS to design and implement scalable data pipelines and ML workflows. You’ll mentor teammates, review code, and help shape architecture in a fast‑paced, collaborative setting.

The role requires strong hands-on experience in big data, distributed systems, and cloud platforms, with a proven track record delivering robust data products for customers.