Senior Data Engineer (Python, PySpark, AWS)

Capital One; United States; On-site; USD 197,000 - 225,000; Full time

Job summary

Capital One is seeking a Data Engineer 4 to push the frontier of cloud-based data platforms. You’ll collaborate with agile teams to design, build, test, and deploy scalable data solutions using Python, Spark, and AWS, shaping the data pipeline architecture across the enterprise.

This role emphasizes mentorship, data modeling, and implementing secure data practices on lakehouse platforms such as Databricks and Snowflake, with a strong focus on performance, reliability, and regulatory compliance.

Qualifications

• Bachelor's degree in Computer Science or related quantitative field.
• At least 4 years of application development experience.
• At least 2 years of distributed data experience.
• At least 2 years of SQL experience.
• At least 2 years of Python/Java/Scala programming experience.
• At least 2 years of data pipeline design and development.
• At least 1 year of data modeling and end-to-end data solutions.

Responsibilities

• Collaborate with Agile teams to design, develop, test, and support data solutions.
• Influence developers, data analysts, and data scientists on data platforms and architectures.
• Utilize Python and Spark with SQL/NoSQL databases and cloud data warehousing (Databricks, Snowflake).
• Contribute to data security and governance practices (encryption, access control).
• Design scalable data pipelines and lakehouse architecture for large data volumes.

Skills

Python

Spark

SQL

AWS

Databricks

Education

Bachelor's degree in Computer Science or related field

Tools

EMR

Glue

Databricks

Snowflake

MongoDB

DynamoDB

Job description

Capital One is seeking a Data Engineer 4 to push the frontier of cloud-based data platforms. You’ll collaborate with agile teams to design, build, test, and deploy scalable data solutions using Python, Spark, and AWS, shaping the data pipeline architecture across the enterprise.

This role emphasizes mentorship, data modeling, and implementing secure data practices on lakehouse platforms such as Databricks and Snowflake, with a strong focus on performance, reliability, and regulatory compliance.