Lead Data Engineer: Build Large-Scale Data Pipelines & AI

Capital One; Chicago, Northern (IL, KY); Hybrid; USD 179,000 - 205,000; Full time

Job summary

Capital One is seeking a Lead Data Engineer in Chicago, IL. You will help design large-scale data pipelines, collaborate with Agile teams, and mentor peers while leveraging Python, Spark, AWS, Snowflake, and Airflow to enable data-driven marketing decisions for millions of customers.

The role emphasizes full-stack, cloud-based solutions, real-time data, and high-performance engineering in a fast-paced environment with a focus on innovation and impact.

Qualifications

• Bachelor's degree required.
• 4+ years of application development experience.
• 2+ years of big data technologies experience.
• 1+ year of cloud computing experience (AWS/Azure/GCP).

Responsibilities

• Collaborate with Agile teams to design, develop, test and support technical solutions across full-stack tools.
• Work with developers experienced in ML, distributed microservices and full-stack systems.
• Utilize Java, Scala, Python and SQL with RDBMS/NoSQL and cloud data warehousing.
• Mentor others and stay updated on tech trends and communities.
• Collaborate with product managers to deliver cloud-based solutions for millions of users.
• Perform unit tests and code reviews to ensure performance and quality.

Skills

Python

SQL

Java

Scala

Spark

Airflow

NoSQL

RDBMS

Git

Education

Bachelor's degree in CS or related field

Tools

AWS

Azure

GCP

Databricks

Snowflake

Redshift

Job description

Capital One is seeking a Lead Data Engineer in Chicago, IL. You will help design large-scale data pipelines, collaborate with Agile teams, and mentor peers while leveraging Python, Spark, AWS, Snowflake, and Airflow to enable data-driven marketing decisions for millions of customers.

The role emphasizes full-stack, cloud-based solutions, real-time data, and high-performance engineering in a fast-paced environment with a focus on innovation and impact.