Senior Data Engineer Lead – Cloud, AI & Data Platforms

Capital One Group; New York (NY); On-site; USD 251,000 - 286,000; Full time

Job summary

Capital One seeks a Data Engineer 5 (Senior Manager IC) in New York to lead end-to-end data initiatives, drive cloud-first data solutions, and mentor a team across data engineers, analysts, and scientists. You will shape architecture and collaborate with product managers and software engineers to deliver scalable data platforms.

You will work with Python, Spark, Databricks, and Snowflake in a fast-paced, iterative environment, influencing decisions on data pipelines, governance, and performance.

Qualifications

• Bachelor's degree or higher in Computer Science or a related quantitative field.
• At least 6 years of application development experience.
• At least 4 years of distributed data experience and SQL.
• At least 4 years programming in Python/Java/Scala.
• Experience designing data pipelines and data modeling.

Responsibilities

• Collaborate with Agile teams to design, develop, test, and support technical solutions.
• Lead and mentor developers, data analysts, and data scientists in data engineering practices.
• Design, build, and optimize cloud-based data pipelines and platforms for scalability and reliability.

Skills

Python

SQL

Spark

Databricks

Snowflake

NoSQL

Airflow

Dagster

Education

Bachelor's Degree or higher in Computer Science or related field

Master's Degree (preferred)

Tools

EMR

Kubernetes

Job description

Capital One seeks a Data Engineer 5 (Senior Manager IC) in New York to lead end-to-end data initiatives, drive cloud-first data solutions, and mentor a team across data engineers, analysts, and scientists. You will shape architecture and collaborate with product managers and software engineers to deliver scalable data platforms.

You will work with Python, Spark, Databricks, and Snowflake in a fast-paced, iterative environment, influencing decisions on data pipelines, governance, and performance.