Senior Data Engineer: Cloud Data Pipelines & GenAI

Capital One Group; New York, Northern (NY, KY); Hybrid; USD 215,000 - 246,000; Full time

Job summary

Capital One is seeking a Data Engineer 4 to design, build, and deploy scalable, cloud-first data solutions and real-time pipelines. You will lead cross-functional teams and ensure governance, data quality, and secure data handling while delivering high-impact data products across Adtech channels.

The role emphasizes modern tech stacks including Python, SQL, Spark, Kafka, Snowflake, and Databricks, with opportunities to apply GenAI tooling to accelerate delivery.

Qualifications

• Bachelor's degree or higher in Computer Science or related quantitative field.
• At least 4 years of application development experience.
• At least 2 years in distributed data workloads.
• At least 2 years of SQL experience.
• At least 2 years in Python, Java, or Scala.
• At least 2 years in data pipeline design and development.
• At least 1 year with relational and non-relational data models.

Responsibilities

• Architect and build scalable, cloud-first data solutions and real-time pipelines.
• Provide technical leadership and mentoring to cross-functional Agile teams.
• Leverage Python, SQL, Scala, Spark and cloud platforms like Databricks and Snowflake.
• Ensure data governance, security, quality, and permissible use across platforms.
• Collaborate with product managers and engineers to deliver robust data products.

Skills

Python

AWS

Spark

Kafka

SQL

Snowflake

Databricks

GenAI

Education

Bachelor's degree or higher in CS or related field

Tools

EMR

AWS Glue

MapReduce

Job description

Capital One is seeking a Data Engineer 4 to design, build, and deploy scalable, cloud-first data solutions and real-time pipelines. You will lead cross-functional teams and ensure governance, data quality, and secure data handling while delivering high-impact data products across Adtech channels.

The role emphasizes modern tech stacks including Python, SQL, Spark, Kafka, Snowflake, and Databricks, with opportunities to apply GenAI tooling to accelerate delivery.