Senior Data Engineering Lead | GCP, Spark & ETL

American Family Insurance; Madison (WI); On-site; USD 131,000 - 220,000; Full time

Job summary

American Family Insurance, based in Madison, WI, seeks a Lead Data Engineer to guide an engineering team in delivering scalable data solutions. You will oversee data pipelines, ensure quality and privacy, and collaborate with architecture and development teams to drive reusable, robust code.

Experts in GCP, SQL/NoSQL, and Python ETL are required, with strong knowledge of Hadoop/Spark and data modeling. Portfolios include security and operational excellence within insurance data contexts.

Qualifications

• Demonstrated customer-driven solutions or service experience.
• Must have GCP experience for this position.
• Strong SQL/NoSQL knowledge with various data stores.
• Extensive Python for ETL and data-driven solutions.
• Able to generalise code with design patterns.
• High-quality, reusable code and libraries.
• Experience with Hadoop or Spark for distributed processing.
• Knowledge of ML toolkits such as sklearn, SparkML, or H2O.
• Solid data understanding in data-rich industries like insurance/finance.
• Experience with dimensional modeling and star schemas.
• Strong DB internals understanding (indexes, logs, transactions).
• Experience with IaC tools like Docker, CloudFormation, Terraform.
• Experience with CI/CD workflows (Jenkins, CI/CD, Git).
• Experience with web services.

Responsibilities

• Lead an engineering team to meet project deadlines and priorities.
• Supervise data engineering team members and activities.
• Ensure data quality, security, privacy, and integrity across the lifecycle.
• Document critical workflows and operational aspects.
• Develop understanding of data sources, granularity, and availability.
• Provide technical oversight to architecture and development teams.
• Create scalable code to load and manipulate data in the warehouse.
• Facilitate communication with project teams and stakeholders.

Skills

GCP experience

SQL/NoSQL

Python ETL

Data modeling

Big data (Hadoop/Spark)

Open source ML toolkits

Data governance/security/privacy

Infrastructure as code

Docker

CloudFormation / Terraform

CI/CD (Jenkins, Git)

Web services consumption

Tools

Docker

CloudFormation

Terraform

Jenkins

Git

Job description

American Family Insurance, based in Madison, WI, seeks a Lead Data Engineer to guide an engineering team in delivering scalable data solutions. You will oversee data pipelines, ensure quality and privacy, and collaborate with architecture and development teams to drive reusable, robust code.

Experts in GCP, SQL/NoSQL, and Python ETL are required, with strong knowledge of Hadoop/Spark and data modeling. Portfolios include security and operational excellence within insurance data contexts.