Senior Data Engineer - Python & AWS, Real-Time Cloud

Capital One; Chicago (IL); On-site; USD 179,400 - 204,700; Full time

Job summary

Capital One in Chicago seeks a Lead Data Engineer to design, build, and scale data solutions across cloud and on‑prem environments. You will collaborate with Agile teams, mentor engineers, and drive technical excellence in Python, AWS, and big data stacks.

This role emphasizes real‑time data, streaming, and data warehousing with Redshift or Snowflake, plus NoSQL stores. You’ll partner with product managers to deliver robust, scalable services for millions of users.

Qualifications

• Bachelor’s degree or higher in a related field.
• At least 4 years of experience in application development (internship experience does not apply).
• At least 2 years of experience in big data technologies.
• At least 1 year experience with cloud computing (AWS, Microsoft Azure, Google Cloud).

Responsibilities

• Collaborate with Agile teams to design, develop, test, implement, and support technical solutions.
• Work with developers on machine learning, distributed microservices, and full‑stack systems.
• Utilize languages like Java, Scala, Python and databases including RDBMS and NoSQL; use cloud data warehousing services.
• Mentor team members and stay current with technology trends while building robust cloud‑based solutions.

Skills

Python

SQL

Scala

Java

Big data

Education

Bachelor’s Degree

Tools

MapReduce

Hadoop

Hive

EMR

Kafka

Spark

Gurobi

MySQL

MongoDB

Cassandra

Redshift

Snowflake

UNIX/Linux

Job description

Capital One in Chicago seeks a Lead Data Engineer to design, build, and scale data solutions across cloud and on‑prem environments. You will collaborate with Agile teams, mentor engineers, and drive technical excellence in Python, AWS, and big data stacks.

This role emphasizes real‑time data, streaming, and data warehousing with Redshift or Snowflake, plus NoSQL stores. You’ll partner with product managers to deliver robust, scalable services for millions of users.