Lead Data Engineer — Python, PySpark & AWS

Capital One; McLean (VA); On-site; USD 197,300 - 225,100; Full time

Job summary

Capital One is hiring a Lead Data Engineer in McLean, VA to design, develop, and operate cloud‑based data solutions. You will work with ML, distributed microservices, and full‑stack systems using Python, PySpark, AWS, and leading data warehousing tools.

The role emphasizes collaboration with Agile teams, mentorship, and delivering robust, high‑performant data platforms that empower financial services for millions of customers.

Qualifications

• Bachelor’s Degree.
• At least 4 years of application development experience.
• At least 2 years of big data technologies experience.
• At least 1 year of cloud computing experience (AWS, Azure, or Google Cloud).
• 7+ years of application development with Python, SQL, Scala, or Java.
• 4+ years with a public cloud (AWS, Azure, or Google Cloud).
• 4+ years with distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, MySQL).
• 4+ years experience with real‑time data and streaming applications.
• 4+ years with NoSQL (Mongo, Cassandra).
• 4+ years data warehousing experience (Redshift or Snowflake).
• 2+ years with Agile engineering practices.

Responsibilities

• Collaborate with Agile teams to design, develop, test, implement, and support solutions.
• Work with developers in ML, distributed microservices, and full‑stack systems.
• Utilize Python, SQL, Scala, Java and data warehousing in cloud environments.
• Mentor engineers and stay current with tech trends and communities.
• Partner with product managers to deliver cloud‑based customer experiences.
• Perform unit tests and code reviews for well‑designed, performant code.

Skills

Python

SQL

Scala

Java

Spark

Education

Bachelor's Degree

Tools

Hive

EMR

MapReduce

Hadoop

Kafka

Gurobi

MySQL

Redshift

Snowflake

Job description

Capital One is hiring a Lead Data Engineer in McLean, VA to design, develop, and operate cloud‑based data solutions. You will work with ML, distributed microservices, and full‑stack systems using Python, PySpark, AWS, and leading data warehousing tools.

The role emphasizes collaboration with Agile teams, mentorship, and delivering robust, high‑performant data platforms that empower financial services for millions of customers.