Senior Data Engineer: Real-Time Data Pipelines & AI

New York Life Insurance Company; New York (NY); Hybrid; USD 100,000 - 143,000; Full time

Benefits offered by this job

Discretionary bonus

Leave programs

Adoption assistance

Student loan repayment programs

Job summary

A leading insurance provider in New York is looking for a Data Engineer to enhance its analytics capabilities. This position involves building and maintaining data pipelines, ensuring high data quality, and collaborating with data scientists on analytics initiatives. Ideal candidates will have 3-7 years of experience with dbt, AWS Glue, and a strong analytical skill set. Compensation ranges from $100,000 to $143,000, with a hybrid work model offering flexibility.

Qualifications

• 3-7+ years of experience in data engineering and analytics.
• Experience with AWS Glue and ETL processes.
• Strong knowledge of dbt data transformation tools.

Responsibilities

• Build and maintain dbt data pipelines.
• Implement real-time data ingestion using PySpark and AWS Glue.
• Collaborate on data quality and availability with data scientists.

Skills

Building and maintaining dbt data models

AWS Glue proficiency

Databricks knowledge

Kafka and streaming data pipelines

Analytical thinking

Problem-solving skills

Team collaboration

Communication skills

Tools

PySpark

SQL

Git

Job description

A leading insurance provider in New York is looking for a Data Engineer to enhance its analytics capabilities. This position involves building and maintaining data pipelines, ensuring high data quality, and collaborating with data scientists on analytics initiatives. Ideal candidates will have 3-7 years of experience with dbt, AWS Glue, and a strong analytical skill set. Compensation ranges from $100,000 to $143,000, with a hybrid work model offering flexibility.