Senior Data Engineer: Python, AWS, Spark & GenAI

Capital One; McLean (VA); On-site; USD 197,000 - 225,000; Full time

Job summary

Capital One in McLean, VA seeks a Data Engineer 4 to design scalable cloud data solutions using Python, Spark, SQL, and Snowflake. You’ll collaborate with agile teams, build streaming and batch pipelines, and mentor others while advancing a major data transformation across the company.

You’ll work with Databricks, Kafka, and GenAI capabilities to deliver end-to-end data platforms with security and compliance in mind, enabling personalized experiences for customers.

Qualifications

• Bachelor's degree in CS or related quantitative field.
• 4+ years of experience in application development.
• 2+ years of experience in distributed data.
• 2+ years of experience with SQL.
• 2+ years of experience in one programming language (Python, Java, or Scala).
• 2+ years of experience in data pipeline design.
• 1+ year of experience in data modeling and designing end-to-end data solutions using relational and NoSQL databases.

Responsibilities

• Collaborate with Agile teams to design, develop, test, and support data solutions.
• Influence teams with ML, distributed microservices, lakehouse architecture, and full-stack systems.
• Utilize Python, Spark, and cloud data warehousing platforms like Databricks and Snowflake.
• Design, build, and deliver cloud data solutions with minimal supervision.
• Architect reusable data engineering patterns for scalable pipelines.
• Communicate data outcomes clearly to stakeholders.

Skills

Python

Spark

SQL

Data pipelines

Distributed data

Cloud platforms

Education

Bachelor's Degree in CS or related field

Tools

Databricks

Snowflake

EMR

Glue

Kafka

Redshift

Job description

Capital One in McLean, VA seeks a Data Engineer 4 to design scalable cloud data solutions using Python, Spark, SQL, and Snowflake. You’ll collaborate with agile teams, build streaming and batch pipelines, and mentor others while advancing a major data transformation across the company.

You’ll work with Databricks, Kafka, and GenAI capabilities to deliver end-to-end data platforms with security and compliance in mind, enabling personalized experiences for customers.