Data Engineer - Scalable Pipelines & AI Data (Hybrid)

New York Life Insurance Co.; Lebanon (NJ); Hybrid; USD 81,000 - 116,000; Full time

Benefits offered by this job

Leave programs

Adoption assistance

Student loan repayment programs

Job summary

New York Life Insurance Co. is seeking a Data Engineer to join the TDAV organization and help deliver clean, reliable data for analytics and AI initiatives across the enterprise.

You will design and maintain scalable data pipelines using dbt, PySpark, AWS Glue, Databricks, and Kafka, while partnering with data scientists, technology teams, and business stakeholders to ensure data quality and governance.

Qualifications

• 2–5+ years of data engineering experience with dbt data models.
• Proficiency in AWS Glue and PySpark for data processing.
• Strong Databricks experience with SQL and notebooks.

Responsibilities

• Build and maintain scalable data models and pipelines.
• Apply software engineering practices to data development.
• Collaborate with data scientists and stakeholders on data governance.
• Translate requirements into development plans and Jira stories.
• Partner with Technology on data strategy and deployments.

Skills

dbt data models

PySpark

AWS Glue

Databricks

Kafka

Communication

Team collaboration

Tools

dbt

Databricks

Kafka

Job description

New York Life Insurance Co. is seeking a Data Engineer to join the TDAV organization and help deliver clean, reliable data for analytics and AI initiatives across the enterprise.

You will design and maintain scalable data pipelines using dbt, PySpark, AWS Glue, Databricks, and Kafka, while partnering with data scientists, technology teams, and business stakeholders to ensure data quality and governance.