Data Engineer Lead

AIG PC Global Services, Inc.; Jersey City (NJ); On-site; USD 125,000 - 135,000; Full time

Benefits offered by this job

Eligible bonus

Comprehensive benefits package

Hybrid flexibility

Total Rewards Program

Job summary

AIG PC Global Services, Inc. seeks a Data Engineer Lead in Jersey City, onsite, to build and own production-grade PySpark and Python data pipelines on AWS, while mentoring engineers and upholding high engineering standards.

You will work in an Agile environment, delivering end-to-end data products, shaping data models and SLAs, and applying lakehouse patterns. GenAI tooling will be used to boost quality and speed of delivery.

Qualifications

• Bachelor’s degree or equivalent in a technical field.
• 7+ years of hands-on data engineering delivering production pipelines end-to-end.
• Strong command of PySpark, Python, and SQL including Spark optimization and query performance tuning in Snowflake.
• Hands-on experience with AWS data services: S3, Glue, EMR, Aurora, Lambda, IAM, and CloudWatch.
• Knowledge of lakehouse patterns and fault-tolerant ingestion in practice.
• Clear communication to explain design decisions to technical and business stakeholders.

Responsibilities

• Design, build, and operate production-grade PySpark and Python pipelines on AWS services with data quality checks and observability.
• Own end-to-end delivery of data products from requirements to production readiness in an Agile/Scrum workflow.
• Translate business requirements into technical designs including pipeline structure, data models, and SLAs.
• Enforce engineering standards, coding conventions, and data quality frameworks; drive code reviews and pair programming.
• Utilize GenAI tooling (Claude Code CLI, Snowflake Cortex) to improve quality and delivery speed.
• Mentor junior engineers and produce ADRs, runbooks, and data dictionaries for documentation.

Skills

PySpark

Python

SQL

Data modeling

Data quality

Observability

GenAI tooling

Education

Bachelor’s degree in Computer Science or related field

Tools

AWS EMR

AWS Glue

Amazon S3

Snowflake

Snowflake Cortex

Claude Code

Claude Code CLI

IAM

CloudWatch

Job description

Build production-ready data platforms at AIG, partnering closely with product and stakeholders while keeping engineering standards high. This hands-on Data Engineer Lead role combines strong pipeline delivery with leadership through reviews, mentorship, and documentation. You will also leverage GenAI tooling, including Claude Code and Snowflake Cortex, as part of day-to-day development to improve code quality and productivity.

Based in Jersey City, NJ (onsite), this role includes a $125,000 to $135,000 base salary range and eligibility for a bonus under the terms of the applicable incentive plan.

What you’ll do

• Design, build, and operate production-grade PySpark and Python pipelines on AWS EMR, Glue, and S3, with integrated data quality checks and observability.
• Own end-to-end delivery of data products, from requirements through production readiness, working in an Agile/Scrum workflow that includes sprint planning, CI/CD, release management, and production readiness.
• Turn business and product requirements into concrete technical designs, including pipeline structure, data models, and defined SLAs.
• Apply architecture and design patterns with clear rationale, evaluating trade-offs and aligning decisions with peers and stakeholders.
• Set and enforce team engineering standards, including coding conventions, data quality frameworks, reusable pipeline patterns, and observability hooks.
• Drive technical quality through thorough code reviews and pairing on difficult problems.
• Use GenAI tooling across the development lifecycle, including Claude Code CLI and Snowflake Cortex, to improve solution quality and delivery speed.
• Collaborate with product owners to refine requirements, surface trade-offs, and push back constructively when scope is not technically workable.
• Communicate design decisions and delivery updates to both technical and non-technical stakeholders.
• Mentor junior engineers through reviews, pairing, and knowledge sharing while building team capability.
• Produce clear technical documentation as a standard deliverable, including Architecture Decision Records (ADRs), runbooks, data dictionaries, and pipeline lineage notes.
• Evaluate and drive adoption of productivity tooling and best practices, setting team norms for safe and effective use.

What you bring

• Bachelor’s degree in Computer Science, Information Systems, Engineering, or a related technical field.
• 7+ years of hands-on data engineering experience delivering production data pipelines end-to-end.
• Strong command of PySpark, Python, and SQL, including Spark optimization, partition management, and query performance tuning in Snowflake.
• Hands-on experience with AWS data services: S3, Glue, EMR, Aurora, Lambda, IAM, and CloudWatch.
• Working knowledge of architecture and design principles such as lakehouse patterns and fault-tolerant ingestion, applied in practice.
• Strong analytical and problem-solving skills, including decomposing complex requirements and reasoning through trade-offs.
• Clear communication skills to explain technical decisions and design rationale to engineering peers and business stakeholders.
• Self-motivated learner, interested in staying current with evolving tools, frameworks, and engineering best practices.

Technologies you’ll work with

PySpark, Python, AWS EMR, AWS Glue, Amazon S3, AWS Aurora, AWS Lambda, IAM, CloudWatch, SQL, Snowflake, Snowflake Cortex, Claude Code, Claude Code CLI, GenAI, Agile/Scrum, CI/CD, Architecture Decision Record (ADR).

Benefits and growth

• Eligible bonus in accordance with the terms of the applicable incentive plan.
• Comprehensive benefits package including health, wellbeing, financial security, and professional development.
• Hybrid flexibility.
• Competitive compensation, with a base salary range of $125,000 to $135,000.
• Created a Total Rewards Program, extending benefits beyond time spent at work.

Culture fit

At AIG, the Data Office is a group of builders, thinkers, and pragmatic problem-solvers. Great data engineering is treated as a team sport, with careful documentation, collaborative design reviews, mentorship across every level, and close partnership with business stakeholders. GenAI is encouraged as a practical force multiplier, not a novelty.

Nice to have

• AWS certification such as Solutions Architect Associate or equivalent.
• Familiarity with insurance or financial services data (claims, policy, risk, or actuarial structures).
• Experience writing ADRs, design notes, or technical documentation in a collaborative engineering team.
• Demonstrated use of GenAI tooling (for example Claude Code CLI or Snowflake Cortex) to improve productivity or solution quality.

In-person collaboration: AIG values in-person collaboration and asks team members to be primarily in the office.