Data Engineer Lead – GenAI-Driven Data Pipelines

American International Group; Parsippany-Troy Hills (NJ); Hybrid; USD 125,000 - 135,000; Full time

Benefits offered by this job

Hybrid work model

Competitive benefits

Job summary

American International Group (AIG) is seeking an experienced Data Engineer Lead to design, build, and operate production-grade data pipelines on AWS, with a focus on reliability, observability, and data quality. You will translate business needs into robust data products and support agile delivery within the Data Office.

The role emphasizes collaboration with product owners and stakeholders, mentoring junior engineers, and adopting GenAI tooling to enhance productivity and code quality across

Qualifications

• 7+ years of hands-on data engineering delivering production data pipelines end-to-end.

Responsibilities

• Design, build, and operate production-grade PySpark and Python pipelines on AWS EMR, Glue, and S3 with integrated data quality checks and observability.

Skills

PySpark

Python

SQL

Spark optimization

Data modeling

Architectural patterns

Analytical thinking

Clear communication

Agile/Scrum

Mentoring

Education

Bachelor's degree in Computer Science / Information Systems / Engineering or related field

Tools

AWS S3

AWS Glue

AWS EMR

AWS Aurora

AWS Lambda

AWS IAM

CloudWatch

Snowflake Cortex

Claude Code CLI

Job description

American International Group (AIG) is seeking an experienced Data Engineer Lead to design, build, and operate production-grade data pipelines on AWS, with a focus on reliability, observability, and data quality. You will translate business needs into robust data products and support agile delivery within the Data Office.

The role emphasizes collaboration with product owners and stakeholders, mentoring junior engineers, and adopting GenAI tooling to enhance productivity and code quality across