Lead Data Engineer — Python, AWS, GenAI

Capital One; San Francisco (CA); On-site; USD 215,000 - 246,000; Full time

Job summary

Capital One is seeking a Lead Data Engineer to drive transformation within Enterprise Platforms Technology (EPTech) and collaborate across Agile teams. You will design, implement and optimize cloud-enabled data pipelines using Java, Python, Scala and SQL, with exposure to Redshift and Snowflake.

You’ll contribute to ML and microservices efforts while mentoring peers and sharing best practices within a vibrant engineering community.

Qualifications

• Bachelor’s Degree in a related field.
• 4+ years of application development experience.
• 2+ years in big data technologies.
• 1+ year of cloud computing experience (AWS/Azure/GCP).

Responsibilities

• Collaborate with Agile teams to design, develop, test, implement, and support technical solutions in full-stack development tools and technologies.
• Work with a team of developers experienced in machine learning, distributed microservices, and full stack systems.
• Utilize Java, Scala, Python and RDBMS/NoSQL databases and cloud-based data warehousing services such as Redshift and Snowflake.
• Mentor other engineers and participate in internal/external technology communities.

Education

Bachelor’s Degree

Tools

Java

Python

Scala

SQL

NoSQL

Redshift

Snowflake

Hadoop

Kafka

Spark

EMR

MySQL

Unix/Linux

Shell scripting

AWS

Azure

GCP

Job description

Capital One is seeking a Lead Data Engineer to drive transformation within Enterprise Platforms Technology (EPTech) and collaborate across Agile teams. You will design, implement and optimize cloud-enabled data pipelines using Java, Python, Scala and SQL, with exposure to Redshift and Snowflake.

You’ll contribute to ML and microservices efforts while mentoring peers and sharing best practices within a vibrant engineering community.