Senior Data Engineer — Python, PySpark & AWS

Capital One Group; McLean (VA); Hybrid; USD 197,000 - 225,000; Full time

Job summary

Capital One is seeking a Data Engineer 4 to design and implement cloud-native data pipelines using Python, Spark, and AWS. You will collaborate with agile teams to build scalable data solutions across distributed systems, data warehouses, and lakehouse architectures like Databricks and Snowflake.

The role emphasizes hands-on development, data security, and mentoring other engineers while delivering robust data products for insights and customer impact.

Qualifications

• Bachelor's degree or higher in Computer Science or related quantitative field.
• 2+ years of experience in distributed data processing and SQL.
• 2+ years of programming with Python, Java, or Scala.
• Experience with data modeling and end-to-end data solutions.

Responsibilities

• Design, develop, test, and support cloud-first data solutions.
• Build and maintain scalable data pipelines and platforms.
• Architect data pipelines with focus on performance and reliability.
• Collaborate with product managers, engineers, and data scientists.
• Mentor other data engineers and promote best practices.

Skills

Python

SQL

Spark/PySpark

Distributed data processing

Education

Bachelor's degree or higher in Computer Science or related quantitative field

Quantitative field background

Tools

AWS

Databricks

Snowflake

Redshift

EMR

Glue

Databricks

Airflow

Dagster

Job description

Capital One is seeking a Data Engineer 4 to design and implement cloud-native data pipelines using Python, Spark, and AWS. You will collaborate with agile teams to build scalable data solutions across distributed systems, data warehouses, and lakehouse architectures like Databricks and Snowflake.

The role emphasizes hands-on development, data security, and mentoring other engineers while delivering robust data products for insights and customer impact.