Lead Data Engineer | GCP, ETL & Big Data

American Family Insurance; Phoenix (AZ); On-site; USD 131,000 - 220,000; Full time

Job summary

American Family Insurance is seeking a senior data engineering lead in Phoenix, AZ to architect and drive big data applications within our platform. You will guide an engineering team, ensure data quality and security, and collaborate with development and business stakeholders.

The role emphasizes GCP, Python-based ETL workflows, and distributed data processing with Hadoop/Spark, aligning with our data-driven products and insurance domain knowledge.

Qualifications

• Must have GCP experience and data engineering background.
• Proven ability to deliver customer-driven data solutions and services.
• Strong SQL/NoSQL and multiple data stores experience.
• Extensive Python for ETL and data pipelines.
• Ability to generalize code with reusable design patterns.
• Write robust, high-quality code and contribute to libraries.
• Experience with Hadoop or Spark for distributed processing.
• Knowledge of ML toolkits such as sklearn, SparkML, or H2O.
• Solid data understanding with insurance/financial domain context.
• Knowledge of dimensional modeling and star schemas.
• Deep understanding of database internals and transactions.
• Experience with IaC and containerized infrastructure.
• Familiarity with Jenkins/CI-CD workflows and Git.
• Experience building and consuming web services.

Responsibilities

• Leads an engineering team to meet project deadlines and priorities.
• Supervises data engineering team members and activities.
• Ensures data quality, security, privacy, and integrity across the lifecycle.
• Documents critical workflows and operational support.
• Develops deep understanding of data sources, granularity, and limits.
• Provides technical oversight for scalable, reusable data solutions.
• Creates maintainable, scalable code to load/manipulate data in the warehouse.
• Facilitates communication with project teams and stakeholders.

Skills

GCP experience

SQL/NoSQL knowledge

Python ETL

Design patterns

Reusable code

Hadoop/Spark

ML toolkits (sklearn/SparkML/H2O)

Insurance/financial data domain

Data modeling (dimensional, star)

DB internals (indexes, logs, TX)

IaC tools (Docker, CloudFormation, TF)

CI/CD and Git

Web services consumption

Tools

Docker

CloudFormation

Terraform

Jenkins

CI/CD

Git

Job description

American Family Insurance is seeking a senior data engineering lead in Phoenix, AZ to architect and drive big data applications within our platform. You will guide an engineering team, ensure data quality and security, and collaborate with development and business stakeholders.

The role emphasizes GCP, Python-based ETL workflows, and distributed data processing with Hadoop/Spark, aligning with our data-driven products and insurance domain knowledge.