Senior Data Engineer - Python, Spark & AWS (ETL)

Capital One Group; Richmond, Northern (VA, KY); Hybrid; USD 179,000 - 205,000; Full time

Benefits offered by this job

Health benefits

Competitive compensation

Inclusive workplace

Job summary

Capital One is seeking a Lead Data Engineer to advance data solutions in a fast-paced, collaborative environment. You will design, build, test, and support services using Python, Spark, and AWS, while mentoring peers and contributing to ML and big data initiatives.

The role emphasizes real-time data processing, cloud-based data warehousing (Redshift/Snowflake), and cross-team collaboration with product managers to enhance financial empowerment for customers.

Qualifications

• Bachelor’s degree required.
• At least 4 years of application development experience.
• At least 2 years in big data technologies.
• At least 1 year of cloud experience (AWS/Azure/GCP).
• Strong SQL and data warehousing knowledge.

Responsibilities

• Collaborate with Agile teams to design, develop, test, and support technical solutions.
• Work with a team experienced in ML, distributed microservices, and full stack systems.
• Utilize Java, Scala, Python and SQL/NoSQL databases; leverage Redshift or Snowflake for data warehousing.
• Mentor teammates and stay updated on new technologies via communities.
• Deliver cloud-based solutions that empower users and improve experiences.
• Perform unit tests and code reviews to ensure quality and performance.

Skills

Team collaboration

Mentoring

Problem solving

Communication

Education

Bachelor’s Degree

Tools

Python

Spark

AWS

ETL

Java

Scala

RDBMS

NoSQL

Redshift

Snowflake

MapReduce

Hadoop

Hive

EMR

Kafka

Gurobi

MySQL

Cloud

Agile practices

Job description

Capital One is seeking a Lead Data Engineer to advance data solutions in a fast-paced, collaborative environment. You will design, build, test, and support services using Python, Spark, and AWS, while mentoring peers and contributing to ML and big data initiatives.

The role emphasizes real-time data processing, cloud-based data warehousing (Redshift/Snowflake), and cross-team collaboration with product managers to enhance financial empowerment for customers.