Data Engineer Lead

American International Group; Northern (KY); Hybrid; USD 125,000 - 135,000; Full time

Job summary

American International Group is seeking a Data Engineer Lead within the Data Office to design, build, and operate production-grade data pipelines using PySpark, Python, and SQL on AWS. You will own end-to-end delivery, define data models and SLAs, and uphold engineering standards.

The role includes mentoring engineers and producing clear architecture documentation. In addition, you will collaborate with stakeholders, leverage GenAI tooling to improve productivity, and communicate design

Qualifications

• 7+ years of hands-on data engineering experience delivering production data pipelines end-to-end.
• Bachelor's degree in CS/IS/Engineering or related field.

Responsibilities

• Design, build, and operate production-grade PySpark and Python pipelines on AWS services.
• Own end-to-end delivery of data products from requirements to production.
• Translate business requirements into concrete technical designs, defining pipeline structure, data models, and SLAs.
• Set and enforce engineering standards across the team.
• Conduct thorough code reviews and pair with engineers on hard problems.
• Leverage GenAI tooling to write better code and improve solution quality.
• Collaborate with product owners to refine requirements and communicate decisions clearly.
• Mentor junior engineers and document architecture decisions and runbooks.

Skills

Strong communication

Analytical thinking

Self-motivated learner

Education

Bachelor's degree in Computer Science, Information Systems, Engineering, or related field

Tools

PySpark

Python

SQL

Snowflake

AWS (S3, Glue, EMR, Aurora, Lambda, IAM, CloudWatch)

GenAI tooling (Claude Code CLI, Snowflake Cortex)

Job description

Data Engineer Lead AIG | Data Office · Data Engineering

About The Role

The Data Engineer Lead is a hands‑on engineering role within AIG's Data Engineering organization. You will design, build, and operate production‑grade data pipelines and platforms — and bring sound engineering judgment to every solution you deliver. We actively leverage GenAI tooling — including Claude Code and Snowflake Cortex — as a genuine productivity multiplier across the development lifecycle. We want someone who is motivated to learn, keeps up with a fast‑moving space, and continuously finds new ways to put these tools to work. You will collaborate closely with the product owner, business stakeholders, and fellow engineers. Strong communication and analytical skills matter here as much as technical depth — the ability to ask the right questions, think through trade‑offs clearly, and explain your reasoning is part of the job.

Responsibilities

Design, build, and operate production‑grade PySpark and Python pipelines on AWS EMR, Glue, and S3 with integrated data quality checks and observability. Own end‑to‑end delivery of data products from requirements through production, working within an Agile/Scrum framework — including sprint planning, CI/CD, release management, and production readiness. Translate business and product requirements into concrete technical designs, defining pipeline structure, data models, and SLAs. Demonstrate a sound understanding of architecture and design patterns, evaluate trade‑offs of design decisions, and articulate clear rationale to peers and stakeholders. Set and enforce engineering standards — coding conventions, data quality frameworks, reusable pipeline patterns, and observability hooks — across the team. Conduct thorough code reviews and pair with engineers on hard problems, raising the technical quality of the whole team. Leverage GenAI tooling — Claude Code CLI, Snowflake Cortex — throughout the development lifecycle to write better code faster and improve solution quality. Collaborate with product owners to refine requirements, surface trade‑offs, and push back constructively when scope is technically unworkable. Communicate design decisions and delivery updates clearly to both engineering peers and non‑technical stakeholders. Mentor junior engineers through review, pairing, and knowledge sharing, building team capability alongside product delivery. Produce clear technical documentation — Architecture Decision Record (ADR), runbooks, data dictionaries, and pipeline lineage notes — as a standard part of delivery, not an afterthought. Evaluate and drive adoption of productivity tooling and best practices, setting team norms for safe and effective use.

Experience / Skills Required

Bachelor's degree in Computer Science, Information Systems, Engineering, or a related technical field. 7+ years of hands‑on data engineering experience with a consistent track record delivering production data pipelines end‑to‑end. Strong command of PySpark, Python, and SQL — including Spark optimization, partition management, and query performance tuning in Snowflake. Hands‑on experience with AWS data services: S3, Glue, EMR, Aurora, Lambda, IAM, and CloudWatch. Working knowledge of architecture and design principles — Lakehouse patterns, data modeling, fault‑tolerant ingestion — with demonstrated ability to apply them in practice. Strong analytical and problem‑solving skills: able to decompose complex requirements, reason through design trade‑offs, and arrive at practical, well‑justified solutions. Clear communicator — able to explain technical decisions and design rationale in plain language to both engineering peers and business stakeholders. Self‑motivated learner with a genuine interest in staying current with evolving tools, frameworks, and engineering best practices.

Preferred / Nice To Have

AWS certification — Solutions Architect Associate or equivalent. Domain familiarity with insurance or financial services data — claims, policy, risk, or actuarial structures. Experience writing ADRs, design notes, or technical documentation in a collaborative engineering team. Demonstrated use of GenAI tooling (e.g. Claude Code CLI or Snowflake Cortex) to improve productivity or solution quality.

Our Culture

AIG's Data Office is a team of builders, thinkers, and pragmatic problem‑solvers. We move with the urgency of a technology company and the accountability of a global insurer. Great data engineering is a team sport here. We write documentation as carefully as we write code, review designs together, mentor across every level, and keep business stakeholders close to the work. We lean into GenAI not as a novelty but as a genuine force multiplier — and we want people who share that mindset. AIG is committed to a diverse and inclusive workforce and encourages candidates from all backgrounds to apply. We offer competitive compensation, comprehensive benefits, and hybrid flexibility.

“The base salary range for this position is $125,000-135,000 and the position is eligible for a bonus in accordance with the terms of the applicable incentive plan. Your actual compensation will be dependent on your skills, experience, and qualifications. If your compensation expectations are above the posted range, we still encourage you to apply—your unique background matters to us. In addition, we’re proud to offer a range of competitive benefits, a summary of which can be viewed here: https://sprcdn-assets.sprinklr.com/248/7cae7257-96d1-4207-bc2c-e1b8447826e3-220371966/2026_AIG_Benefits_Overview.pdf .

Functional Area: DT - Data

AIG PC Global Services, Inc. As a global risk leader with deep industry expertise and innovative solutions to smartly manage risk, AIG enables our clients’ growth in ways they never thought possible. We also do the same for our colleagues, because we know our people are our greatest strength – the source of every insight, every idea and every innovation. When we're working as one team to do what's right for our colleagues and our communities, we can achieve excellence together. We encourage colleagues to give back to the causes they care most about, supporting these efforts through our Volunteer Time Off and Matching Grants Programs.

AIG provides equal opportunity to all qualified individuals regardless of race, color, religion, age, gender, gender expression, national origin, veteran status, disability or any other legally protected categories. AIG is committed to working with and providing reasonable accommodations to job applicants and employees with disabilities. If you believe you need a reasonable accommodation, please send an email to [e-mail].

Additional information about AIG can be found at www.aig.com | YouTube | Instagram | LinkedIn.

Join our Talent Community.