Data Engineer 4: GenAI & Real-Time Cloud Pipelines

Capital One National Association; New York, Northern (NY, KY); Hybrid; USD 215,000 - 246,000; Full time

Job summary

Capital One National Association in New York is seeking a Data Engineer 4 to design and deploy cloud-first data solutions, focusing on lakehouse architectures and real-time streaming pipelines. You will lead cross-functional teams, implement data governance and security, and work with technologies like Python, SQL, Spark, Kafka, Snowflake, and Databricks to enable hyper-personalized customer experiences.

The role offers a competitive NY salary range and opportunities to apply GenAI tooling to

Qualifications

• Bachelor's degree or higher in CS or a related quantitative field.
• At least 4 years of application development experience.
• At least 2 years of distributed data experience.
• At least 2 years of SQL experience.
• At least 2 years of experience in one programming language (Python, Java, or Scala).

Responsibilities

• Architect and build resilient, scalable, cloud-first data solutions and real-time pipelines.
• Provide technical leadership and mentorship to cross-functional teams.
• Leverage Python, SQL, Spark, and cloud platforms like Databricks and Snowflake.
• Maintain data governance, quality, and security standards.
• Collaborate with product managers and engineers to deliver robust data solutions.
• Communicate complex concepts clearly to both technical and non-technical stakeholders.

Skills

Python

AWS

Spark

Kafka

SQL

Snowflake

Databricks

GenAI

Education

Bachelor's Degree or higher in Computer Science or related quantitative field

Tools

Databricks

Snowflake

EMR

Hadoop

Job description

Capital One National Association in New York is seeking a Data Engineer 4 to design and deploy cloud-first data solutions, focusing on lakehouse architectures and real-time streaming pipelines. You will lead cross-functional teams, implement data governance and security, and work with technologies like Python, SQL, Spark, Kafka, Snowflake, and Databricks to enable hyper-personalized customer experiences.

The role offers a competitive NY salary range and opportunities to apply GenAI tooling to