Lead Data Engineer, Cloud Data & Real-Time Pipelines

Capital One National Association; New York (NY); On-site; USD 215,200 - 245,600; Full time

Job summary

Capital One is seeking a Lead Data Engineer in New York, NY to architect and deliver scalable data pipelines and cloud-based solutions. You will work across Agile teams to design, test, and implement data movement platforms using Snowflake, Databricks, Python and SQL in a fast-paced environment.

You will guide a team of engineers, mentor peers, and stay ahead of industry trends while ensuring data quality, governance, and secure data handling in a high-impact, customer-focused setting.

Qualifications

• Bachelor’s Degree in a relevant field.
• At least 4 years of experience in application development.
• At least 2 years of experience in big data technologies.
• At least 1 year experience with cloud computing (AWS, Microsoft Azure, Google Cloud).

Responsibilities

• Collaborate with Agile teams to design, develop, test, implement, and support technical solutions using data movement tools and technologies.
• As a lead developer, work with a team of developers with deep experience in data movement, distributed computing, and full-stack systems.
• Utilize Python, SQL and open source RDBMS/NoSQL databases and cloud-based data warehousing services such as Snowflake and Databricks.
• Optimize information systems for end-users and downstream application consumers with sound data design.
• Mentor other members of the engineering community and stay current with tech trends.
• Collaborate with product managers to deliver robust cloud-based solutions for millions of Americans.
• Perform unit tests and code reviews to ensure quality and performance.

Skills

Data modeling

Data quality

Agile engineering

Cloud computing

Mentoring

Big data

Education

Bachelor's degree

Master's degree

Tools

Snowflake

Databricks

Python

SQL

NoSQL databases

AWS

Azure

GCP

MapReduce

Hadoop

Hive

EMR

Kafka

Spark

MySQL

UNIX/Linux

Shell scripting

Job description

Capital One is seeking a Lead Data Engineer in New York, NY to architect and deliver scalable data pipelines and cloud-based solutions. You will work across Agile teams to design, test, and implement data movement platforms using Snowflake, Databricks, Python and SQL in a fast-paced environment.

You will guide a team of engineers, mentor peers, and stay ahead of industry trends while ensuring data quality, governance, and secure data handling in a high-impact, customer-focused setting.