Lead Data Engineer: Cloud, Real-Time Data & ML

Capital One; New York (NY); On-site; USD 215,000 - 246,000; Full time

Job summary

Capital One in New York is seeking a Lead Data Engineer to help design, build, and optimize data solutions at scale. You will collaborate across Agile teams to deliver cloud-based data platforms and streaming applications.

You’ll work with Java, Scala, Python and technologies like Redshift, Snowflake, Hadoop, Spark, and Kafka, and mentor others while advancing the data-driven transformation across the company.

Qualifications

• Bachelor's Degree in a related field.
• At least 4 years of experience in application development.
• At least 2 years of experience in big data technologies.
• At least 1 year experience with cloud computing (AWS, Azure, GCP).

Responsibilities

• Collaborate with Agile teams to design, develop, test, implement, and support technical solutions.
• Work with developers experienced in ML, distributed microservices, and full-stack systems.
• Utilize Java, Scala, Python and Open Source RDBMS/NoSQL databases; use cloud data warehousing like Redshift and Snowflake.
• Mentor other engineering team members and stay current on tech trends.

Skills

Java

Python

Scala

SQL

Big data

Agile engineering

Education

Bachelor’s Degree

Tools

Redshift

Snowflake

Hadoop

Kafka

Spark

MySQL

MongoDB

Cassandra

EMR

Job description

Capital One in New York is seeking a Lead Data Engineer to help design, build, and optimize data solutions at scale. You will collaborate across Agile teams to deliver cloud-based data platforms and streaming applications.

You’ll work with Java, Scala, Python and technologies like Redshift, Snowflake, Hadoop, Spark, and Kafka, and mentor others while advancing the data-driven transformation across the company.