Lead Data Engineer - Cloud, Spark & Big Data

Capital One; McLean (VA); On-site; USD 197,300 - 225,100; Full time

Job summary

Capital One is seeking a Lead Data Engineer to design, develop, test, and support Big Data solutions across Agile teams. You will work with Java, Scala, Python, and cloud data warehouses such as Redshift/Snowflake, mentoring engineers and driving performance improvements.

The role emphasizes collaboration with product managers to deliver cloud-enabled experiences for customers and includes rigorous unit testing and reviews to ensure robust, scalable systems.

Qualifications

• Bachelor’s degree in a related field.
• At least 4 years of application development experience.
• At least 2 years in big data technologies.
• At least 1 year cloud experience (AWS/Azure/GCP).

Responsibilities

• Collaborate with Agile teams to design, develop, test, implement, and support Big Data solutions.
• Work with developers experienced in ML, distributed microservices, and ETL.
• Utilize Java, Scala, Python and cloud data warehouses like Redshift and Snowflake.
• Mentor engineers and stay current with tech trends in the industry.
• Collaborate with product managers to deliver cloud-based solutions serving millions.
• Perform unit tests and code reviews to ensure well-designed, efficient code.

Skills

Python

SQL

Scala

Java

Big data technologies

Real-time data streaming

NoSQL databases

UNIX/Linux

Agile methodologies

Cloud computing (AWS, Azure, GCP)

Distributed data systems (Hadoop,Spark

Kafka

Education

Bachelor’s degree

Tools

Redshift

Snowflake

Open Source RDBMS

MongoDB

Cassandra

Hadoop

Spark

EMR

MapReduce

Hive

Job description

Capital One is seeking a Lead Data Engineer to design, develop, test, and support Big Data solutions across Agile teams. You will work with Java, Scala, Python, and cloud data warehouses such as Redshift/Snowflake, mentoring engineers and driving performance improvements.

The role emphasizes collaboration with product managers to deliver cloud-enabled experiences for customers and includes rigorous unit testing and reviews to ensure robust, scalable systems.