Senior Data Engineer: Python, AWS, Spark, Kafka, GenAI

Capital One National Association; New York (NY); On-site; USD 215,000 - 246,000; Full time

Job summary

Capital One is seeking a Data Engineer 4 to design and deploy cloud-first data solutions, emphasizing data modeling, governance, and scalable pipelines. You’ll lead cross-functional teams using Python, SQL, Spark, Kafka, and modern platforms like Databricks and Snowflake to deliver hyper-personalized experiences.

You will mentor engineers, collaborate with product managers, and ensure secure, high-quality data flows across Adtech platforms, enabling impactful customer insights and rapid delivery

Qualifications

• Bachelor's degree in Computer Science or related quantitative field.
• At least 4 years of experience in application development.
• At least 2 years of experience with SQL.
• At least 2 years of programming in Python/Java/Scala.
• At least 2 years in data pipeline design and development.
• At least 1 year in data modeling for relational and non-relational stores.

Responsibilities

• Architect and build scalable, cloud-first data solutions and real-time pipelines.
• Provide technical leadership and mentorship to cross-functional teams.
• Leverage Python, SQL, Spark; work with Databricks and Snowflake.
• Implement data governance, security, and quality controls.
• Collaborate with product managers and engineers to deliver robust data products.
• Communicate technical concepts clearly to diverse stakeholders.
• Ensure unit testing, code reviews, and system observability.

Skills

Python

SQL

Spark

Kafka

Snowflake

Databricks

GenAI

Education

Bachelor's Degree in Computer Science or related field

Master’s Degree (preferred)

Tools

AWS

Azure

GCP

Databricks

Snowflake

Hadoop

Airflow

Job description

Capital One is seeking a Data Engineer 4 to design and deploy cloud-first data solutions, emphasizing data modeling, governance, and scalable pipelines. You’ll lead cross-functional teams using Python, SQL, Spark, Kafka, and modern platforms like Databricks and Snowflake to deliver hyper-personalized experiences.

You will mentor engineers, collaborate with product managers, and ensure secure, high-quality data flows across Adtech platforms, enabling impactful customer insights and rapid delivery